-- Logs begin at Sun 2022-08-07 21:36:41 UTC, end at Sun 2022-08-07 23:00:09 UTC. -- Aug 07 22:18:16.736216 dv-cinder-850321-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 07 22:18:16.745167 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 07 22:18:16.747186 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Aug 7 22:18:16 2022] *** Aug 07 22:18:16.747186 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 07 22:18:16.747186 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 07 22:18:16.747186 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: nodename: dv-cinder-850321-3-iscsi Aug 07 22:18:16.747186 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: machine: x86_64 Aug 07 22:18:16.747186 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: clock source: unix Aug 07 22:18:16.747186 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: pcre jit disabled Aug 07 22:18:16.747186 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: detected number of CPU cores: 4 Aug 07 22:18:16.747186 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: current working directory: / Aug 07 22:18:16.747186 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: detected binary path: /usr/bin/uwsgi-core Aug 07 22:18:16.747186 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: your processes number limit is 47680 Aug 07 22:18:16.747767 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: your memory page size is 4096 bytes Aug 07 22:18:16.747767 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: detected max file descriptor number: 2048 Aug 07 22:18:16.747767 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: lock engine: pthread robust mutexes Aug 07 22:18:16.747767 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: thunder lock: enabled Aug 07 22:18:16.747767 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 07 22:18:16.747767 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 07 22:18:16.787248 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: Python main interpreter initialized at 0x55d56beae9c0 Aug 07 22:18:16.787248 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: python threads support enabled Aug 07 22:18:16.787248 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: your server socket listen backlog is limited to 100 connections Aug 07 22:18:16.787248 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: your mercy for graceful operations on workers is 90 seconds Aug 07 22:18:16.787584 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: mapped 403077 bytes (393 KB) for 2 cores Aug 07 22:18:16.787658 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: *** Operational MODE: preforking *** Aug 07 22:18:16.787933 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: *** uWSGI is running in multiple interpreter mode *** Aug 07 22:18:16.787933 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: spawned uWSGI master process (pid: 80358) Aug 07 22:18:16.788151 dv-cinder-850321-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 07 22:18:16.796055 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: spawned uWSGI worker 1 (pid: 80370, cores: 1) Aug 07 22:18:16.796055 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: spawned uWSGI worker 2 (pid: 80371, cores: 1) Aug 07 22:18:16.796055 dv-cinder-850321-3-iscsi devstack@keystone.service[80358]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 07 22:18:22.360687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 22:18:22.360687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: from cryptography.utils import int_from_bytes Aug 07 22:18:22.361429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 22:18:22.361429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: from cryptography.utils import int_from_bytes Aug 07 22:18:22.362626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 22:18:22.362626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: from cryptography.utils import int_from_bytes Aug 07 22:18:22.363062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 22:18:22.363062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: from cryptography.utils import int_from_bytes Aug 07 22:18:22.412178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 22:18:22.412368 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 22:18:22.412545 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] command line args: [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 22:18:22.412994 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 22:18:22.413314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ================================================================================ {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 22:18:22.413452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 22:18:22.413722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] admin_token = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.413940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 22:18:22.414046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] config_dir = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.414153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.414340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] config_source = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.414518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] debug = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.414720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.414986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] command line args: [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 22:18:22.415086 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.415270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.415525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.415703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 22:18:22.415794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.415994 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.416203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_config_append = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.416311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ================================================================================ {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 22:18:22.416400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.416543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_dir = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.416721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_file = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.416959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.417153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] admin_token = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.417250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.417444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.417693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.417872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.417998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] config_dir = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.418080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.418278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.418502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.418603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.418770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.419008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.419164 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] config_source = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.419259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.419554 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] debug = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.419652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.419879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.420118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.420231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.420420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.420718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.420914 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.421094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] publish_errors = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.421293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.421393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.421543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.421722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.421968 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.422146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.422242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.422421 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] standard_threads = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.422650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.422863 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.422955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.423052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.423229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] use_journal = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.423405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] use_json = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.423578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] use_stderr = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.423752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] use_syslog = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.423912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.424000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.424250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.424463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_config_append = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.424560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.424737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.424978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.425096 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.425226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.425465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.425644 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_dir = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.425734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.425937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.426150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_file = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.426242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.426433 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.426686 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.426927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.427017 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.427173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.427410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.427660 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.427749 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.427907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.428165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.428425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.428515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.428678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.428925 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.429135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.429228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.429404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.429668 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.429759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.429921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.430176 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.430273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.430406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.430639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.430871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.431013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.431106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.431346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.431607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.431817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.431916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.432082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.432334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.432563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.432657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.432809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.433059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.433252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.433350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.433521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.433757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.433969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.434068 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.434214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.435062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.435166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.435531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.435632 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.435980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.436138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.436257 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.436497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.436739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.436909 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.437013 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.437193 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.437430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.437678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.437910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.438149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.438298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.438386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.438609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.438829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] publish_errors = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.438929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.439070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.439303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.439556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.439791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.439947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.440128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.440267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.440530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.440769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.441002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.441228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.441319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.441467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.441740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.441958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.442043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.442215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.442461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.442698 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.442915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.443009 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.443153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.443403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.443643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.443862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.443946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.444214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.444459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.444679 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] standard_threads = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.444788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.444919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.445029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.445188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.445380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.445459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.445568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] use_journal = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.445650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.445758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] use_json = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.446777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.446887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] use_stderr = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.446887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] use_syslog = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.447005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 22:18:22.447171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.447951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.448035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.448177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.448272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.448515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.448763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.448867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.449002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.449176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.449283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.449531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.449703 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.449872 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.449872 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.450253 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.450253 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.450551 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.450551 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.450853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.451059 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.451217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.451393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.451564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.451740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.451910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.external = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.451910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.452258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.452258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.452560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.452730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.452907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.452907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.password = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.453192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.453308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.token = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.453447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.453563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.453676 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.453836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.453955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.454074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.454188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.454300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.454425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.454591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.454722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.454865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.454988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.455108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.455230 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.455345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.455457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.455576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.455711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.455833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.456078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.456198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.456312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.456424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.456537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.456656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.456782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.456905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.457037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.457153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.457266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.457378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.457490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.457603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.457725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.457725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.457960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.458095 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.458210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.458323 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.458445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.458617 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.458742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.458878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.459000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.459115 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.459237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.459352 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.459465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.459577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.459689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.459802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.459922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.460080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.460216 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.460351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.460485 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.460630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.460765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.460883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.461003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.461117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.461236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.461361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.461476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.461589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.461701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.461918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.462034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.462155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.462276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.462397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.462513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.462666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.462840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.462967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.463083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.463186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.463306 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.463428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.463598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.463725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.463841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.463955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.464097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.464213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.464386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.464502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.464620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.464746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.464861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.464975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.465110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.465232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.465347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.465469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.466146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.466269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.466383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.466503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.466619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.466740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.466969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.467082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.467267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.467384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.467510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.467627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.467741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.467855 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.467969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.468131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.468262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.468379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.468493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.468607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.468720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.468833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.468953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.469111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.469228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.469350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.469465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.469579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.469712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.469828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.469942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.470056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.470176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.470292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.470478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.470603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.470718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.470839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.470961 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.471081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.471203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.471203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.471420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.471533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.471646 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.471759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.471929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.472171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.472301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.472418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.472621 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.472752 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.472878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.472878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.473092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.473205 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.473358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.473512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.473641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.473794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.473911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.474026 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.474139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.474252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.474374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.external = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.474491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.474611 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.474727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.474841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.474955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.475081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.475196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.475310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.password = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.475430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.475546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.token = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.475659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.475781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.475781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.475996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.475996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.476226 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.476226 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.476456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.476456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.476672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.476835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.476975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.476975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.477226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.477341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.477454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.477567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.477681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.477681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.477933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.478049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.478163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.478267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.478379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.478531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.478651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.478765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.478879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.478993 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.479106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.479232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.479347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.479461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.479574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.479688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.479803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.479924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.480077 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.480195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.480310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.480444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.480571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.480695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.480853 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.480969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.481083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.481300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.481428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.481552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.481684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.481808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.481924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.482062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.482176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.482298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.482413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.482528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.482641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.482755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.482869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.482990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.483105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.483220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.483333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.483453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.483568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.483689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.483804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.483920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.484047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.484163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.484276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.484401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.484517 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.484631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.484631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.484845 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.484963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.485085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.485085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.485301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.485414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.485528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.485641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.485763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.485878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.485992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.486105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.486218 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.486332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.486453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.486568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.486682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.486796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.486910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.487023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.487146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] role.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.487261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.487375 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] role.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.487513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.487645 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.487775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.487912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.488071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.488211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.488343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.488473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.488614 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.488746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.488876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.489005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.489149 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.489279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.489425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.489556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.489685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.489815 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.489944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.490073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.490212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.490384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.490557 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.490692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.490823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.491012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.491187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.491355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.491486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.491486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.491783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.491946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.492111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.492282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.492418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.492577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.492740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.492875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.493014 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.493158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.493310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.493485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.493650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.494058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.494058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.494371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.494531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.494701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.494896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.495033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.495239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.495393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.495605 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.495740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.495871 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.496079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.496251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.496416 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.496553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.496713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.497423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.497561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.497700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.497832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.498050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.498201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.498382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.498518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.498659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.498790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.498942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.499111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.499243 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.499372 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.499511 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.499643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.499772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.499931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.500084 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.500228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.500370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.500536 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.500688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.500872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.501068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.501200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.501368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.501504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.501666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.501801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.501931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.502069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.502224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.502368 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.502499 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.502629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.502781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.502948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.503082 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.503246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.503382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.503512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.503671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.503820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.503953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.504143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.connection = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.504317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.504633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.504768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.504899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.505038 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.505190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.505331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.505493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.505626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.505756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.505895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.506049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.506184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.506343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.506487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.506618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.506757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.506910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.507109 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.507274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.507408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.507538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.507676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.507807 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.507938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.508107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.508244 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.508406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.508552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.508706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.508841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.508994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509426 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509557 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 22:18:22.509839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.509839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.511007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.511007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.511007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] role.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.511007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] role.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.511007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.511007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.511007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.511007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.511007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.512131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.512131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.512131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.512131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.512131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.512131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.512131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.512131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.512131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.513498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.513498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.513498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.513498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.513498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.513498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.513498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.513498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.513498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.514713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.514713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.514713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.514713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.514713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.514713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.514713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.514713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.514713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.515914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.515914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.515914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.515914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.515914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.515914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.515914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.515914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.515914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.517150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.517150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.517150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.517150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.517150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.517150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.517150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.517150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.517150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.518279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.518279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.connection = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.518279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.518279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.518279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.519031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.519031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.519031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.519415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.519415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.519683 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.519841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.520066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.520307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.520546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.520782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.521026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.521256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.521494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.521728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 22:18:22.521897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 22:18:22.531679 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.531951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.532159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.542083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.547523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.547753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.547979 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.557927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.570977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.571209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.577239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.577472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.628242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.628420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.681857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.682078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.694431 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.694664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.757767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.757987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.758080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.758205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.758288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.758423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.758509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.758637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.758720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.758846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.758928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.759056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.782736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.782846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.782972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.783102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.783205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.783287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.783397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.783493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.783636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.783727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.783852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.783933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.784087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.784173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.795928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.796072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.808702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.808815 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.808927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.809011 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.809149 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.809233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.809508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.809594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.809809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.809905 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.810032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.810190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.810272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.810417 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 22:18:22.827582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.827787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 22:18:22.828159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.828271 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.828355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 22:18:22.828481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 22:18:22.828623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.828820 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.828902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.829020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 22:18:22.829127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.829299 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.829389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.829555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.829834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.830087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.840814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.841085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.841350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.841624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.841706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.841899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.842168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.842446 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.848524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.849349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.851557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.851840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.852148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.852411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.852494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.852696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.852777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.852952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.853035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.853221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.853304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.853495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.853585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.853759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.853837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.854029 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.854109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.854293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.854383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.854560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.854839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.855114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.877839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.879332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.879423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.879578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.880825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.881120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.884159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.884441 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.884718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.885689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.885972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.886251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.891768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.892077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.893293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.893578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.896149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.896439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.896721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.897013 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.897296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.897582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.897671 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.897850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.897972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.898133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.898214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.898411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.898685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.898956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.899226 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.899499 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.914580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.914858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.915959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.916260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.918856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.922127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.922427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.922527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.925737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.926034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.926875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.930068 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.930399 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.933224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.933525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.933643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.933813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.934097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.934400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.934698 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.936767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.937070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.937410 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.937646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.937925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.938204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.951257 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.951546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.951813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.952107 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.954576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.954871 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.955139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.955402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.958938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.960541 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.960820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.962400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.963631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.963916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.963999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.964232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.964327 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.964522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.964799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.967606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.967999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.968328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.968619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.968897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.976896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.977174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.977450 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.977725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.978001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.978275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.978546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.980222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.980507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.980789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.981071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.981474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.981725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.982004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:22.995697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.998803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:22.999162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:23.001750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.002177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:23.003328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.004852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:23.005132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.005215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.009124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.009214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.009214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:23.009214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.009214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.011772 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.014041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:23.015867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.017007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.017301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.017543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:23.017644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.017838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.020471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.020761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.021042 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.021325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.025908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:23.026199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.026478 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.029951 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:23.030236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.030516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.034018 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:23.034301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.034577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.034848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.035116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.035395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.035666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.035932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.036334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.036616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.036885 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.037156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.037430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.037523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 22:18:23.037689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.037854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.038122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.038402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.038676 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.038959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.039232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.039518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.039897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.040194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.040479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.040751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.041021 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.041282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 22:18:23.071401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d56beae9c0 pid: 80371 (default app) Aug 07 22:18:23.074714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8e55b-ccf2-43ab-a3ee-c6fdaca263ad None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:23.075075 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8e55b-ccf2-43ab-a3ee-c6fdaca263ad None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:23.075166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d56beae9c0 pid: 80370 (default app) Aug 07 22:18:23.075348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8e55b-ccf2-43ab-a3ee-c6fdaca263ad None None] PATH_INFO: `/v3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:23.079628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Sun Aug 7 22:18:23 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:31.693688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edb9e5-27bb-4e5c-803d-633700b6e9c7 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:31.694207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edb9e5-27bb-4e5c-803d-633700b6e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:31.694207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edb9e5-27bb-4e5c-803d-633700b6e9c7 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:31.698965 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:18:31.705242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fd4fa-d03c-46db-9a5b-1d972c073218 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:31.705681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fd4fa-d03c-46db-9a5b-1d972c073218 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:31.705766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fd4fa-d03c-46db-9a5b-1d972c073218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:31.785454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: INFO dbcounter [None req-c32fd4fa-d03c-46db-9a5b-1d972c073218 None None] Registered counter for database keystone Aug 07 22:18:31.807470 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG oslo_db.sqlalchemy.engines [None req-c32fd4fa-d03c-46db-9a5b-1d972c073218 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80371) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 22:18:31.820479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writer thread running {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 22:18:32.025528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG passlib.handlers.bcrypt [None req-c32fd4fa-d03c-46db-9a5b-1d972c073218 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80371) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 07 22:18:32.025941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG passlib.handlers.bcrypt [None req-c32fd4fa-d03c-46db-9a5b-1d972c073218 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80371) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 07 22:18:32.091315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c32fd4fa-d03c-46db-9a5b-1d972c073218 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:32.093056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c32fd4fa-d03c-46db-9a5b-1d972c073218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:32.255246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 07 22:18:32.406783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edb9e5-27bb-4e5c-803d-633700b6e9c7 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:32.476768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: INFO dbcounter [None req-84edb9e5-27bb-4e5c-803d-633700b6e9c7 None None] Registered counter for database keystone Aug 07 22:18:32.500310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG oslo_db.sqlalchemy.engines [None req-84edb9e5-27bb-4e5c-803d-633700b6e9c7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80370) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 22:18:32.505710 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writer thread running {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 22:18:32.624575 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edb9e5-27bb-4e5c-803d-633700b6e9c7 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:32.628821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6009087-4e9a-4b33-ae35-596e75731c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:32.637813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6009087-4e9a-4b33-ae35-596e75731c66 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:32.637813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6009087-4e9a-4b33-ae35-596e75731c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:32.637813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6009087-4e9a-4b33-ae35-596e75731c66 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:32.638807 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6009087-4e9a-4b33-ae35-596e75731c66 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:32.691631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b6009087-4e9a-4b33-ae35-596e75731c66 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 07 22:18:32.691911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6009087-4e9a-4b33-ae35-596e75731c66 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:32.702295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Sun Aug 7 22:18:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:18:32.708212 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fd4fa-d03c-46db-9a5b-1d972c073218 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:32.723101 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fd4fa-d03c-46db-9a5b-1d972c073218 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:32.727547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e2227-2503-4e72-a852-33a99786f149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:32.728683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e2227-2503-4e72-a852-33a99786f149 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:32.728965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e2227-2503-4e72-a852-33a99786f149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:32.729237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e2227-2503-4e72-a852-33a99786f149 admin admin] PATH_INFO: `/v3/projects/ff949bd3888c436789a339dd0a2e16fc` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:32.730659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e2227-2503-4e72-a852-33a99786f149 admin admin] RBAC: Authorizing `identity:get_project(project_id=ff949bd3888c436789a339dd0a2e16fc)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:32.781766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: WARNING keystone.common.rbac_enforcer.enforcer [None req-324e2227-2503-4e72-a852-33a99786f149 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 07 22:18:32.782108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e2227-2503-4e72-a852-33a99786f149 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:32.786771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Sun Aug 7 22:18:32 2022] GET /identity/v3/projects/ff949bd3888c436789a339dd0a2e16fc => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:34.752226 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78179dff-530b-4ad9-9271-3904a8e3de27 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:34.752802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78179dff-530b-4ad9-9271-3904a8e3de27 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:34.752802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78179dff-530b-4ad9-9271-3904a8e3de27 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:34.756906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:34.762291 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331181e-d9e9-4c34-a202-fde9e76b55df None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:34.762786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331181e-d9e9-4c34-a202-fde9e76b55df None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:34.762898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331181e-d9e9-4c34-a202-fde9e76b55df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:34.783852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1331181e-d9e9-4c34-a202-fde9e76b55df None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:34.785540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1331181e-d9e9-4c34-a202-fde9e76b55df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:34.793922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 07 22:18:34.815998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78179dff-530b-4ad9-9271-3904a8e3de27 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:34.828659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78179dff-530b-4ad9-9271-3904a8e3de27 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:34.832858 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9a559-a5ab-4bf6-9661-b0fb9d62c973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:34.833445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9a559-a5ab-4bf6-9661-b0fb9d62c973 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:34.833728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9a559-a5ab-4bf6-9661-b0fb9d62c973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:34.833999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9a559-a5ab-4bf6-9661-b0fb9d62c973 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:34.841586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9a559-a5ab-4bf6-9661-b0fb9d62c973 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:34.843341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9a559-a5ab-4bf6-9661-b0fb9d62c973 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:34.851868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Sun Aug 7 22:18:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 07 22:18:34.857582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331181e-d9e9-4c34-a202-fde9e76b55df None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:34.866479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331181e-d9e9-4c34-a202-fde9e76b55df None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:34.871075 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2cfb3-b864-4384-ba5d-f677c9331479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:34.872320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2cfb3-b864-4384-ba5d-f677c9331479 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:34.872596 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2cfb3-b864-4384-ba5d-f677c9331479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:34.872869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2cfb3-b864-4384-ba5d-f677c9331479 admin admin] PATH_INFO: `/v3/users/620b714c359944a6b1d8649f781a6815` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:34.874217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2cfb3-b864-4384-ba5d-f677c9331479 admin admin] RBAC: Authorizing `identity:get_user(user_id=620b714c359944a6b1d8649f781a6815)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:34.876041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2cfb3-b864-4384-ba5d-f677c9331479 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:34.877545 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Sun Aug 7 22:18:34 2022] GET /identity/v3/users/620b714c359944a6b1d8649f781a6815 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:36.917823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d334a1d-6a60-4f17-bc07-d108b427cd99 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:36.918992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d334a1d-6a60-4f17-bc07-d108b427cd99 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:36.918992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d334a1d-6a60-4f17-bc07-d108b427cd99 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:36.920295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:36.925369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4e8c0-b5c5-4868-8c9e-2190c8b69909 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:36.925797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4e8c0-b5c5-4868-8c9e-2190c8b69909 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:36.925927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4e8c0-b5c5-4868-8c9e-2190c8b69909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:36.944463 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8df4e8c0-b5c5-4868-8c9e-2190c8b69909 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:36.945644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8df4e8c0-b5c5-4868-8c9e-2190c8b69909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:37.000148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6bdb-f510-4f07-b2e2-3fefd6a7f266 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:37.000429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6bdb-f510-4f07-b2e2-3fefd6a7f266 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:37.000708 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6bdb-f510-4f07-b2e2-3fefd6a7f266 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:37.002029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:18:37.007173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89fe7a-95ea-41f9-b684-3f01bf5a9e8f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:37.007456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89fe7a-95ea-41f9-b684-3f01bf5a9e8f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:37.007731 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89fe7a-95ea-41f9-b684-3f01bf5a9e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:37.026664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:37.049269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4e8c0-b5c5-4868-8c9e-2190c8b69909 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:37.052855 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG passlib.handlers.bcrypt [None req-7e89fe7a-95ea-41f9-b684-3f01bf5a9e8f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80370) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 07 22:18:37.053272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG passlib.handlers.bcrypt [None req-7e89fe7a-95ea-41f9-b684-3f01bf5a9e8f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80370) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 07 22:18:37.072233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:37.099485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7e89fe7a-95ea-41f9-b684-3f01bf5a9e8f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:37.101119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7e89fe7a-95ea-41f9-b684-3f01bf5a9e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:37.152191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4e8c0-b5c5-4868-8c9e-2190c8b69909 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:37.186856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b3e50-9d8d-4b78-b9fb-bb19f6284c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:37.187426 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b3e50-9d8d-4b78-b9fb-bb19f6284c12 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:37.187710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b3e50-9d8d-4b78-b9fb-bb19f6284c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:37.187988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b3e50-9d8d-4b78-b9fb-bb19f6284c12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:37.199003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b3e50-9d8d-4b78-b9fb-bb19f6284c12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:37.201313 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-037b3e50-9d8d-4b78-b9fb-bb19f6284c12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:37.208636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:37.209765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:18:37.210599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:18:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:18:37.215671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89fe7a-95ea-41f9-b684-3f01bf5a9e8f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:37.224155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b3e50-9d8d-4b78-b9fb-bb19f6284c12 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:37.306052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:37.317618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89fe7a-95ea-41f9-b684-3f01bf5a9e8f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:37.324148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:37.324873 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b3e50-9d8d-4b78-b9fb-bb19f6284c12 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:37.355378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07d401-db28-4a4b-8a04-fefc50227e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:37.356905 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d401-db28-4a4b-8a04-fefc50227e30 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:37.357232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d401-db28-4a4b-8a04-fefc50227e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:37.357529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d401-db28-4a4b-8a04-fefc50227e30 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:37.358326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07d401-db28-4a4b-8a04-fefc50227e30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:37.358635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07d401-db28-4a4b-8a04-fefc50227e30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:37.360188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07d401-db28-4a4b-8a04-fefc50227e30 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:37.361303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fab68b-4efa-4367-8c11-f28ef26a2382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:37.361901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fab68b-4efa-4367-8c11-f28ef26a2382 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:37.362180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fab68b-4efa-4367-8c11-f28ef26a2382 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:37.362452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fab68b-4efa-4367-8c11-f28ef26a2382 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:37.372984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 8/15] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:18:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:37.378640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07d401-db28-4a4b-8a04-fefc50227e30 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:37.484031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07d401-db28-4a4b-8a04-fefc50227e30 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:37.520827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8e529-711e-4e70-976d-e36e6e0c6bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:37.521680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8e529-711e-4e70-976d-e36e6e0c6bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:37.521963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8e529-711e-4e70-976d-e36e6e0c6bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:37.522249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8e529-711e-4e70-976d-e36e6e0c6bd6 None admin] PATH_INFO: `/v3/users/620b714c359944a6b1d8649f781a6815` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:37.523752 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8e529-711e-4e70-976d-e36e6e0c6bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=620b714c359944a6b1d8649f781a6815)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:37.525251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8e529-711e-4e70-976d-e36e6e0c6bd6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:37.526981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 9/16] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:37 2022] GET /identity/v3/users/620b714c359944a6b1d8649f781a6815 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:37.532324 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8e529-711e-4e70-976d-e36e6e0c6bd6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:37.536963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fab68b-4efa-4367-8c11-f28ef26a2382 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:37.538449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fab68b-4efa-4367-8c11-f28ef26a2382 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:37.565936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:37.617691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:37.632902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8e529-711e-4e70-976d-e36e6e0c6bd6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:37.637576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 8/17] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:18:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 22:18:37.643022 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fab68b-4efa-4367-8c11-f28ef26a2382 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:37.667182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b250c-ad01-4047-8521-e53080dd5362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:37.667882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b250c-ad01-4047-8521-e53080dd5362 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:37.668852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b250c-ad01-4047-8521-e53080dd5362 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:37.669132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b250c-ad01-4047-8521-e53080dd5362 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:37.670253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b250c-ad01-4047-8521-e53080dd5362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:37.671690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b250c-ad01-4047-8521-e53080dd5362 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:37.673252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 10/18] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:18:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:37.678833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b250c-ad01-4047-8521-e53080dd5362 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:37.742980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fab68b-4efa-4367-8c11-f28ef26a2382 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:37.774638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b250c-ad01-4047-8521-e53080dd5362 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:37.776157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:37.776272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8b264-241f-461b-8e13-dddef4bd2273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:37.776783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8b264-241f-461b-8e13-dddef4bd2273 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:37.777062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8b264-241f-461b-8e13-dddef4bd2273 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:37.777333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8b264-241f-461b-8e13-dddef4bd2273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:37.787259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8b264-241f-461b-8e13-dddef4bd2273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:37.788859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8b264-241f-461b-8e13-dddef4bd2273 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-98b8b264-241f-461b-8e13-dddef4bd2273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:18:37.804578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:37.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:18:37.806233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:18:37.806233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:18:37.806233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:18:37.806233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 9/19] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:18:37.810213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719015b4-a758-41f2-8c59-3def7151905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:37.810818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719015b4-a758-41f2-8c59-3def7151905a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:37.811097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719015b4-a758-41f2-8c59-3def7151905a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:37.811372 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719015b4-a758-41f2-8c59-3def7151905a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:37.812137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719015b4-a758-41f2-8c59-3def7151905a None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.domain.id=default, user.id=620b714c359944a6b1d8649f781a6815) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:37.812457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719015b4-a758-41f2-8c59-3def7151905a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:37.813358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8b264-241f-461b-8e13-dddef4bd2273 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:37.813895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719015b4-a758-41f2-8c59-3def7151905a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:37.822157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 11/20] 10.222.0.30 () {56 vars in 1322 bytes} [Sun Aug 7 22:18:37 2022] GET /identity/v3/role_assignments?user.id=620b714c359944a6b1d8649f781a6815&scope.domain.id=default&role.id=108664305d2546949f7327b65d4cb667 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:37.849363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:37.922073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8b264-241f-461b-8e13-dddef4bd2273 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:37.959310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a8a94-370c-462f-bab3-db9ec6bbade4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:37.959972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a8a94-370c-462f-bab3-db9ec6bbade4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:37.960226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a8a94-370c-462f-bab3-db9ec6bbade4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:37.960498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a8a94-370c-462f-bab3-db9ec6bbade4 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:37.961150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a8a94-370c-462f-bab3-db9ec6bbade4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:37.961451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a8a94-370c-462f-bab3-db9ec6bbade4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:37.962854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a8a94-370c-462f-bab3-db9ec6bbade4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:37.980558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 10/21] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:39.981618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb70a9-e7e4-4180-b70d-76ef7c70f2db None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:39.982259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb70a9-e7e4-4180-b70d-76ef7c70f2db None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:39.982259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb70a9-e7e4-4180-b70d-76ef7c70f2db None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:39.983577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 12/22] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:39.993127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8acb0e-a57a-48e0-b108-a8fd0589f35b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:39.993892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8acb0e-a57a-48e0-b108-a8fd0589f35b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:39.994584 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8acb0e-a57a-48e0-b108-a8fd0589f35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:40.022908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ec8acb0e-a57a-48e0-b108-a8fd0589f35b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:40.024370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ec8acb0e-a57a-48e0-b108-a8fd0589f35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:40.122037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 11/23] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:40.151434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb70a9-e7e4-4180-b70d-76ef7c70f2db None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:40.215808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c473-4ce1-40b9-9893-c50f0e5b51b5 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:40.216581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c473-4ce1-40b9-9893-c50f0e5b51b5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:40.217056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c473-4ce1-40b9-9893-c50f0e5b51b5 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:40.218624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 12/24] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:18:40.223773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d635cb-a084-46dd-8bf4-1f3ccadcaa1d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:40.224313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d635cb-a084-46dd-8bf4-1f3ccadcaa1d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:40.224805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d635cb-a084-46dd-8bf4-1f3ccadcaa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:40.252405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-61d635cb-a084-46dd-8bf4-1f3ccadcaa1d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:40.255413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-61d635cb-a084-46dd-8bf4-1f3ccadcaa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:40.295046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:40.300369 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb70a9-e7e4-4180-b70d-76ef7c70f2db None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:40.387767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd99e0-9309-4963-98dc-89c9c00be656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:40.388480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd99e0-9309-4963-98dc-89c9c00be656 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:40.388769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd99e0-9309-4963-98dc-89c9c00be656 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:40.389052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd99e0-9309-4963-98dc-89c9c00be656 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:40.406388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 13/25] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:18:40.421078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd99e0-9309-4963-98dc-89c9c00be656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d9fd99e0-9309-4963-98dc-89c9c00be656 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:40.430196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:40.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:18:40.431771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:18:40.434421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 13/26] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:18:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:40.439414 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d635cb-a084-46dd-8bf4-1f3ccadcaa1d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:40.446595 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd99e0-9309-4963-98dc-89c9c00be656 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:40.489908 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:40.630119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d635cb-a084-46dd-8bf4-1f3ccadcaa1d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:40.664001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd99e0-9309-4963-98dc-89c9c00be656 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:40.699075 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e724a8-fcea-45df-8411-020e99042f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:40.700283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e724a8-fcea-45df-8411-020e99042f89 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:40.700571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e724a8-fcea-45df-8411-020e99042f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:40.700844 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e724a8-fcea-45df-8411-020e99042f89 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:40.701531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e724a8-fcea-45df-8411-020e99042f89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:40.701885 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e724a8-fcea-45df-8411-020e99042f89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:40.703261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e724a8-fcea-45df-8411-020e99042f89 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:40.731061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 14/27] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:18:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:40.737165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e724a8-fcea-45df-8411-020e99042f89 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:40.800584 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:40.881192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13338a3c-8a0e-4e31-9375-c1f67537a52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:40.881836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338a3c-8a0e-4e31-9375-c1f67537a52f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:40.882118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338a3c-8a0e-4e31-9375-c1f67537a52f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:40.882392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338a3c-8a0e-4e31-9375-c1f67537a52f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:40.923222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13338a3c-8a0e-4e31-9375-c1f67537a52f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:40.924772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13338a3c-8a0e-4e31-9375-c1f67537a52f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-13338a3c-8a0e-4e31-9375-c1f67537a52f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:18:40.945606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:40.946703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:18:40.947772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:18:40.947772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:18:40.947772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:18:40.950909 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 14/28] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 505 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:40.958424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476b5b0-87a7-4800-83b5-7afc9644185c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:40.958999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476b5b0-87a7-4800-83b5-7afc9644185c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:40.959462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476b5b0-87a7-4800-83b5-7afc9644185c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:40.961424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 15/29] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:40.962406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476b5b0-87a7-4800-83b5-7afc9644185c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:40.989164 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e724a8-fcea-45df-8411-020e99042f89 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:41.032871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae33e41-1aca-489d-8806-f0c32037209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:41.034252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae33e41-1aca-489d-8806-f0c32037209f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:41.034707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae33e41-1aca-489d-8806-f0c32037209f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:41.035148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae33e41-1aca-489d-8806-f0c32037209f None admin] PATH_INFO: `/v3/users/620b714c359944a6b1d8649f781a6815` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:41.037205 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae33e41-1aca-489d-8806-f0c32037209f None admin] RBAC: Authorizing `identity:get_user(user_id=620b714c359944a6b1d8649f781a6815)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:41.039553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae33e41-1aca-489d-8806-f0c32037209f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:41.041952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 15/30] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:40 2022] GET /identity/v3/users/620b714c359944a6b1d8649f781a6815 => generated 247 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:41.044185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fe86f-51f7-4d65-ae98-36e9947cc03c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:41.044651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fe86f-51f7-4d65-ae98-36e9947cc03c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:41.045112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fe86f-51f7-4d65-ae98-36e9947cc03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:41.067903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9d0fe86f-51f7-4d65-ae98-36e9947cc03c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:41.069825 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9d0fe86f-51f7-4d65-ae98-36e9947cc03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:41.070884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:41.098580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476b5b0-87a7-4800-83b5-7afc9644185c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:41.135143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23dfbd-6087-4ac9-bed8-c61bc262ff54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:41.136170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23dfbd-6087-4ac9-bed8-c61bc262ff54 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:41.136696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23dfbd-6087-4ac9-bed8-c61bc262ff54 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:41.137803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23dfbd-6087-4ac9-bed8-c61bc262ff54 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:41.138759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23dfbd-6087-4ac9-bed8-c61bc262ff54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:41.139273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23dfbd-6087-4ac9-bed8-c61bc262ff54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:41.140883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23dfbd-6087-4ac9-bed8-c61bc262ff54 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:41.153023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 16/31] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:41.154368 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664c625-5a98-4324-bf89-3da68c613baa None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:41.154655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664c625-5a98-4324-bf89-3da68c613baa None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:41.154930 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664c625-5a98-4324-bf89-3da68c613baa None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:41.156519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 17/32] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:41.157225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664c625-5a98-4324-bf89-3da68c613baa None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:41.161564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 16/33] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:41.163001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838945ca-1f6b-4ede-9d75-bb4a5f276a3f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:41.163507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838945ca-1f6b-4ede-9d75-bb4a5f276a3f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:41.163981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838945ca-1f6b-4ede-9d75-bb4a5f276a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:41.183456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-838945ca-1f6b-4ede-9d75-bb4a5f276a3f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:41.185246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-838945ca-1f6b-4ede-9d75-bb4a5f276a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:41.188150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:41.257375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 17/34] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:41.258596 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838945ca-1f6b-4ede-9d75-bb4a5f276a3f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:41.287277 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664c625-5a98-4324-bf89-3da68c613baa None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:41.324460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155653d3-3dfb-471a-8260-ec09655189e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:41.324962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155653d3-3dfb-471a-8260-ec09655189e7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:41.325250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155653d3-3dfb-471a-8260-ec09655189e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:41.325526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155653d3-3dfb-471a-8260-ec09655189e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:41.326686 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155653d3-3dfb-471a-8260-ec09655189e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:41.328195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155653d3-3dfb-471a-8260-ec09655189e7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:41.329784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 18/35] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:18:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:41.330530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155653d3-3dfb-471a-8260-ec09655189e7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:41.331866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:41.370370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838945ca-1f6b-4ede-9d75-bb4a5f276a3f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:41.408914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc50fd8-d19f-4040-82ec-ea2a516100ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:41.410130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc50fd8-d19f-4040-82ec-ea2a516100ab None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:41.411379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc50fd8-d19f-4040-82ec-ea2a516100ab None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:41.411890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc50fd8-d19f-4040-82ec-ea2a516100ab None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:41.412821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc50fd8-d19f-4040-82ec-ea2a516100ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:41.414601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc50fd8-d19f-4040-82ec-ea2a516100ab None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:41.423864 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:41.443043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155653d3-3dfb-471a-8260-ec09655189e7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:41.481635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048a92e-4fad-4eb6-b6eb-f5ffc3b6583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:41.482252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048a92e-4fad-4eb6-b6eb-f5ffc3b6583b None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:41.482531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048a92e-4fad-4eb6-b6eb-f5ffc3b6583b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:41.482808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048a92e-4fad-4eb6-b6eb-f5ffc3b6583b None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:41.483480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048a92e-4fad-4eb6-b6eb-f5ffc3b6583b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:41.484911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048a92e-4fad-4eb6-b6eb-f5ffc3b6583b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:41.648751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 18/36] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:18:41 2022] POST /identity/v3/projects => generated 301 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:18:41.650015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc50fd8-d19f-4040-82ec-ea2a516100ab None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:41.722657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 19/37] 10.222.0.30 () {60 vars in 1131 bytes} [Sun Aug 7 22:18:41 2022] POST /identity/v3/roles => generated 219 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:18:41.732128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048a92e-4fad-4eb6-b6eb-f5ffc3b6583b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:41.856712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc50fd8-d19f-4040-82ec-ea2a516100ab None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:41.881180 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:41.903245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8711611-23a0-459c-9dd1-dac36a1ddedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:41.904484 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8711611-23a0-459c-9dd1-dac36a1ddedf None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:41.906569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8711611-23a0-459c-9dd1-dac36a1ddedf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:41.907133 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8711611-23a0-459c-9dd1-dac36a1ddedf None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:41.909041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8711611-23a0-459c-9dd1-dac36a1ddedf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:41.912043 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8711611-23a0-459c-9dd1-dac36a1ddedf None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:41.967907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048a92e-4fad-4eb6-b6eb-f5ffc3b6583b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:41.992886 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 19/38] 10.222.0.30 () {60 vars in 1131 bytes} [Sun Aug 7 22:18:41 2022] POST /identity/v3/roles => generated 225 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:18:41.995570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b40bc-d29a-48ea-8980-00da2381aa2f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:41.995930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b40bc-d29a-48ea-8980-00da2381aa2f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:41.996936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b40bc-d29a-48ea-8980-00da2381aa2f None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:41.998948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 20/39] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:42.001199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ed3e7-a76a-4515-b865-ba0da68fabcc None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:42.001495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ed3e7-a76a-4515-b865-ba0da68fabcc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:42.001750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ed3e7-a76a-4515-b865-ba0da68fabcc None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:42.003261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 21/40] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:42.006515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb4789-8784-475e-89d4-fc86840941e0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:42.006800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb4789-8784-475e-89d4-fc86840941e0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:42.007079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb4789-8784-475e-89d4-fc86840941e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:42.022026 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b08aed-f236-41d4-b97c-5d781fa39d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:42.022776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b08aed-f236-41d4-b97c-5d781fa39d41 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:42.023594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b08aed-f236-41d4-b97c-5d781fa39d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:42.023594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b08aed-f236-41d4-b97c-5d781fa39d41 None admin] PATH_INFO: `/v3/domains/default/users/620b714c359944a6b1d8649f781a6815/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:42.025642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b08aed-f236-41d4-b97c-5d781fa39d41 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=620b714c359944a6b1d8649f781a6815, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:42.027879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b08aed-f236-41d4-b97c-5d781fa39d41 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:42.030025 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-46fb4789-8784-475e-89d4-fc86840941e0 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:42.031885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-46fb4789-8784-475e-89d4-fc86840941e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:42.098197 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 20/41] 10.222.0.30 () {58 vars in 1361 bytes} [Sun Aug 7 22:18:41 2022] PUT /identity/v3/domains/default/users/620b714c359944a6b1d8649f781a6815/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:18:42.109663 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a82aee-3f40-41ca-a3a8-ace93cc624dd None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:42.109955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a82aee-3f40-41ca-a3a8-ace93cc624dd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:42.110234 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a82aee-3f40-41ca-a3a8-ace93cc624dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:42.123617 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 22/42] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:18:42.124958 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:42.144104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e0a82aee-3f40-41ca-a3a8-ace93cc624dd None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:42.144104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e0a82aee-3f40-41ca-a3a8-ace93cc624dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:42.156117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb4789-8784-475e-89d4-fc86840941e0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:42.235964 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 21/43] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:42.259683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a82aee-3f40-41ca-a3a8-ace93cc624dd None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:42.301351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb4789-8784-475e-89d4-fc86840941e0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:42.314456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:42.339847 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffe65d-e9b5-4472-88ed-5e687ead2175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:42.340753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffe65d-e9b5-4472-88ed-5e687ead2175 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:42.341067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffe65d-e9b5-4472-88ed-5e687ead2175 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:42.341346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffe65d-e9b5-4472-88ed-5e687ead2175 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:42.342107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daffe65d-e9b5-4472-88ed-5e687ead2175 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:42.343467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daffe65d-e9b5-4472-88ed-5e687ead2175 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:42.370499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a82aee-3f40-41ca-a3a8-ace93cc624dd None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:42.400621 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:42.405839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8921a-887e-426b-ba42-25ef65063aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:42.406557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8921a-887e-426b-ba42-25ef65063aed None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:42.406955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8921a-887e-426b-ba42-25ef65063aed None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:42.407238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8921a-887e-426b-ba42-25ef65063aed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:42.409012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd8921a-887e-426b-ba42-25ef65063aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:42.410496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd8921a-887e-426b-ba42-25ef65063aed None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:42.412318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 22/44] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:18:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:42.412607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 23/45] 10.222.0.30 () {60 vars in 1131 bytes} [Sun Aug 7 22:18:42 2022] POST /identity/v3/roles => generated 223 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:18:42.417868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8921a-887e-426b-ba42-25ef65063aed None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:42.536256 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8921a-887e-426b-ba42-25ef65063aed None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:42.576553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c99bcc-6a07-45f2-86f3-125b708ed977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:42.577225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c99bcc-6a07-45f2-86f3-125b708ed977 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:42.577519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c99bcc-6a07-45f2-86f3-125b708ed977 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:42.577850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c99bcc-6a07-45f2-86f3-125b708ed977 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:42.578529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c99bcc-6a07-45f2-86f3-125b708ed977 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:42.580002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c99bcc-6a07-45f2-86f3-125b708ed977 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:42.664795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 23/46] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:18:42 2022] POST /identity/v3/projects => generated 312 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:18:44.140400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7062d-44d2-4c9c-a54d-3e01aa2d781f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:44.140907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7062d-44d2-4c9c-a54d-3e01aa2d781f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:44.140988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7062d-44d2-4c9c-a54d-3e01aa2d781f None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:44.142276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 24/47] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:44.147295 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facde1d8-5338-4489-8a4d-f2a9d97eb16f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:44.147702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facde1d8-5338-4489-8a4d-f2a9d97eb16f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:44.147859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facde1d8-5338-4489-8a4d-f2a9d97eb16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:44.165731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-facde1d8-5338-4489-8a4d-f2a9d97eb16f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:44.166933 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-facde1d8-5338-4489-8a4d-f2a9d97eb16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:44.214100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:44.233247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 24/48] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:44.256736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7062d-44d2-4c9c-a54d-3e01aa2d781f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:44.358092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7062d-44d2-4c9c-a54d-3e01aa2d781f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:44.392854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f2aab-80af-496d-a6a2-bbbd5a1bf828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:44.393483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2aab-80af-496d-a6a2-bbbd5a1bf828 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:44.393769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2aab-80af-496d-a6a2-bbbd5a1bf828 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:44.394049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2aab-80af-496d-a6a2-bbbd5a1bf828 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:44.404564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f2aab-80af-496d-a6a2-bbbd5a1bf828 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-1a4f2aab-80af-496d-a6a2-bbbd5a1bf828 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:44.409393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:44.410376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:18:44.411269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 25/49] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:18:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:44.416376 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facde1d8-5338-4489-8a4d-f2a9d97eb16f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:44.516041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facde1d8-5338-4489-8a4d-f2a9d97eb16f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:44.549619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f377a6-6430-4086-b734-00298428aae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:44.550233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f377a6-6430-4086-b734-00298428aae9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:44.550516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f377a6-6430-4086-b734-00298428aae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:44.550794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f377a6-6430-4086-b734-00298428aae9 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:44.551466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f377a6-6430-4086-b734-00298428aae9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:44.551774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f377a6-6430-4086-b734-00298428aae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:44.553196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f377a6-6430-4086-b734-00298428aae9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:44.561341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 25/50] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:18:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:44.566924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f2aab-80af-496d-a6a2-bbbd5a1bf828 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:44.598349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:44.655968 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edcb2f-a491-41ee-8f69-f8e3b3e31766 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:44.656294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edcb2f-a491-41ee-8f69-f8e3b3e31766 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:44.656586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edcb2f-a491-41ee-8f69-f8e3b3e31766 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:44.657940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 26/51] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:18:44.662822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dffa2-7227-4a87-a81b-1a421b6353ef None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:44.663118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dffa2-7227-4a87-a81b-1a421b6353ef None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:44.663413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dffa2-7227-4a87-a81b-1a421b6353ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:44.668728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f2aab-80af-496d-a6a2-bbbd5a1bf828 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:44.681814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-747dffa2-7227-4a87-a81b-1a421b6353ef None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:44.683046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-747dffa2-7227-4a87-a81b-1a421b6353ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:44.703947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195caf0-13d7-459b-a6ab-eb3faf736a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:44.704605 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195caf0-13d7-459b-a6ab-eb3faf736a14 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:44.704890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195caf0-13d7-459b-a6ab-eb3faf736a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:44.705175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195caf0-13d7-459b-a6ab-eb3faf736a14 None admin] PATH_INFO: `/v3/users/620b714c359944a6b1d8649f781a6815` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:44.706308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195caf0-13d7-459b-a6ab-eb3faf736a14 None admin] RBAC: Authorizing `identity:get_user(user_id=620b714c359944a6b1d8649f781a6815)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:44.707812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195caf0-13d7-459b-a6ab-eb3faf736a14 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:44.709436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 26/52] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:44 2022] GET /identity/v3/users/620b714c359944a6b1d8649f781a6815 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:44.714890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195caf0-13d7-459b-a6ab-eb3faf736a14 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:44.724729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:44.751593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 27/53] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:18:44.776176 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dffa2-7227-4a87-a81b-1a421b6353ef None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:44.813208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:44.816902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195caf0-13d7-459b-a6ab-eb3faf736a14 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:44.851033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b93ea4-3a9a-4e95-825f-bcc76b5917a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:44.851723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b93ea4-3a9a-4e95-825f-bcc76b5917a3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:44.852029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b93ea4-3a9a-4e95-825f-bcc76b5917a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:44.852318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b93ea4-3a9a-4e95-825f-bcc76b5917a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:44.853446 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b93ea4-3a9a-4e95-825f-bcc76b5917a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:44.854903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b93ea4-3a9a-4e95-825f-bcc76b5917a3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:44.857052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 27/54] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:18:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:44.862776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b93ea4-3a9a-4e95-825f-bcc76b5917a3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:44.875162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dffa2-7227-4a87-a81b-1a421b6353ef None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:44.908906 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde9f96-6382-4ae2-a1a1-8ce7deece85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:44.909537 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde9f96-6382-4ae2-a1a1-8ce7deece85e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:44.909816 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde9f96-6382-4ae2-a1a1-8ce7deece85e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:44.910089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde9f96-6382-4ae2-a1a1-8ce7deece85e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:44.911215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde9f96-6382-4ae2-a1a1-8ce7deece85e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:44.912687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde9f96-6382-4ae2-a1a1-8ce7deece85e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:44.914245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 28/55] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:18:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:44.920581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde9f96-6382-4ae2-a1a1-8ce7deece85e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:44.962598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b93ea4-3a9a-4e95-825f-bcc76b5917a3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:44.963402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:44.998542 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08e72e-74f4-439c-925b-ebbb7c7b47e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:44.999242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08e72e-74f4-439c-925b-ebbb7c7b47e8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:44.999532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08e72e-74f4-439c-925b-ebbb7c7b47e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:44.999800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08e72e-74f4-439c-925b-ebbb7c7b47e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:45.000581 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08e72e-74f4-439c-925b-ebbb7c7b47e8 None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.domain.id=default, user.id=620b714c359944a6b1d8649f781a6815) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:45.000895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08e72e-74f4-439c-925b-ebbb7c7b47e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:45.002372 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08e72e-74f4-439c-925b-ebbb7c7b47e8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:45.008214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 28/56] 10.222.0.30 () {56 vars in 1322 bytes} [Sun Aug 7 22:18:44 2022] GET /identity/v3/role_assignments?user.id=620b714c359944a6b1d8649f781a6815&scope.domain.id=default&role.id=108664305d2546949f7327b65d4cb667 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:45.020965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde9f96-6382-4ae2-a1a1-8ce7deece85e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:45.059367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe49db0-5463-4852-88c0-8e6ae9a709a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:45.059910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe49db0-5463-4852-88c0-8e6ae9a709a9 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:45.060223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe49db0-5463-4852-88c0-8e6ae9a709a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:45.060503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe49db0-5463-4852-88c0-8e6ae9a709a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:45.061176 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe49db0-5463-4852-88c0-8e6ae9a709a9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:45.062700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe49db0-5463-4852-88c0-8e6ae9a709a9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:45.144800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 29/57] 10.222.0.30 () {60 vars in 1140 bytes} [Sun Aug 7 22:18:44 2022] POST /identity/v3/projects => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:18:47.142490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d54ae4-8f46-4753-85dd-24205a9180c6 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:47.142997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d54ae4-8f46-4753-85dd-24205a9180c6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:47.143080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d54ae4-8f46-4753-85dd-24205a9180c6 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:47.144425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 29/58] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:47.149480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837de752-f1a9-4522-a2a2-838aa669ba24 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:47.149876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837de752-f1a9-4522-a2a2-838aa669ba24 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:47.150039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837de752-f1a9-4522-a2a2-838aa669ba24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:47.169346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-837de752-f1a9-4522-a2a2-838aa669ba24 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:47.170549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-837de752-f1a9-4522-a2a2-838aa669ba24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:47.236510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 30/59] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:47.260281 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d54ae4-8f46-4753-85dd-24205a9180c6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:47.290109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:47.358565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d54ae4-8f46-4753-85dd-24205a9180c6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:47.390555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97094aab-7c61-4c08-b908-4d3fc152307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:47.391166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97094aab-7c61-4c08-b908-4d3fc152307b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:47.391445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97094aab-7c61-4c08-b908-4d3fc152307b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:47.391718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97094aab-7c61-4c08-b908-4d3fc152307b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:47.392855 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97094aab-7c61-4c08-b908-4d3fc152307b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:47.394279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97094aab-7c61-4c08-b908-4d3fc152307b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:47.395812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 30/60] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:18:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:47.401316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837de752-f1a9-4522-a2a2-838aa669ba24 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:47.426662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:47.500902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837de752-f1a9-4522-a2a2-838aa669ba24 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:47.532907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff62b2a-9aa1-42a9-a317-4724c855fe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:47.533508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff62b2a-9aa1-42a9-a317-4724c855fe12 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:47.533796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff62b2a-9aa1-42a9-a317-4724c855fe12 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:47.534072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff62b2a-9aa1-42a9-a317-4724c855fe12 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:47.534711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff62b2a-9aa1-42a9-a317-4724c855fe12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:47.536184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff62b2a-9aa1-42a9-a317-4724c855fe12 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:47.610440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 31/61] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:18:47 2022] POST /identity/v3/users => generated 275 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:18:50.522829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffbb00-01f2-485e-ae0e-dd86b482b77d None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:50.523821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffbb00-01f2-485e-ae0e-dd86b482b77d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:50.524342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffbb00-01f2-485e-ae0e-dd86b482b77d None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:50.526172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 31/62] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:50.532051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbc6c1-1891-4ddd-9290-15eb1430ebd8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:50.532856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbc6c1-1891-4ddd-9290-15eb1430ebd8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:50.533330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbc6c1-1891-4ddd-9290-15eb1430ebd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:50.552492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-adbbc6c1-1891-4ddd-9290-15eb1430ebd8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:50.553999 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-adbbc6c1-1891-4ddd-9290-15eb1430ebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:50.624195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 32/63] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:50.671849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ffbb00-01f2-485e-ae0e-dd86b482b77d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:50.746737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f83ed1-5be8-40c4-9b40-5a31ef8aa68b None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:50.747297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f83ed1-5be8-40c4-9b40-5a31ef8aa68b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:50.747779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f83ed1-5be8-40c4-9b40-5a31ef8aa68b None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:50.749322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 33/64] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:18:50.754460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ec58f-bae2-449f-8c86-827b4ac30269 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:50.754984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ec58f-bae2-449f-8c86-827b4ac30269 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:50.755459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ec58f-bae2-449f-8c86-827b4ac30269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:50.781629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e24ec58f-bae2-449f-8c86-827b4ac30269 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:50.784410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e24ec58f-bae2-449f-8c86-827b4ac30269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:50.792344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:50.851460 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:50.863227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ffbb00-01f2-485e-ae0e-dd86b482b77d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:50.916485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e87f6-1a0b-4100-b4ae-bf6ee6a7b435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:50.918356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e87f6-1a0b-4100-b4ae-bf6ee6a7b435 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:50.919057 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e87f6-1a0b-4100-b4ae-bf6ee6a7b435 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:50.920826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e87f6-1a0b-4100-b4ae-bf6ee6a7b435 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:50.925128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 34/65] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:18:50.940087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e87f6-1a0b-4100-b4ae-bf6ee6a7b435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-3a7e87f6-1a0b-4100-b4ae-bf6ee6a7b435 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:50.945295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:50.946126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:18:50.946888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:18:50.949391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 32/66] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:18:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:50.955145 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ec58f-bae2-449f-8c86-827b4ac30269 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:50.959178 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e87f6-1a0b-4100-b4ae-bf6ee6a7b435 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:51.157167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e87f6-1a0b-4100-b4ae-bf6ee6a7b435 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:51.166841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ec58f-bae2-449f-8c86-827b4ac30269 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:51.195153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c0e9a-944c-41a6-ac04-93706ab211e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:51.196245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c0e9a-944c-41a6-ac04-93706ab211e9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.196713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c0e9a-944c-41a6-ac04-93706ab211e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.197169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c0e9a-944c-41a6-ac04-93706ab211e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.198261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c0e9a-944c-41a6-ac04-93706ab211e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:51.198751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c0e9a-944c-41a6-ac04-93706ab211e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:51.201038 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c0e9a-944c-41a6-ac04-93706ab211e9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:51.210664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 33/67] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:18:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:51.217250 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c0e9a-944c-41a6-ac04-93706ab211e9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:51.239347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da3ee0-08ca-4ac9-b2ab-ef8c6690df6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:51.239988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da3ee0-08ca-4ac9-b2ab-ef8c6690df6a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.240312 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da3ee0-08ca-4ac9-b2ab-ef8c6690df6a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.240587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da3ee0-08ca-4ac9-b2ab-ef8c6690df6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.242221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:51.248836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:51.251876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da3ee0-08ca-4ac9-b2ab-ef8c6690df6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-a3da3ee0-08ca-4ac9-b2ab-ef8c6690df6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:51.256664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:51.257503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:18:51.258288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 35/68] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:18:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:51.259403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dee6e3-0143-435b-a4b8-ed501337f389 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.259688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dee6e3-0143-435b-a4b8-ed501337f389 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.259964 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dee6e3-0143-435b-a4b8-ed501337f389 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.261307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 36/69] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:51.266073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d174349-aaf6-4734-b03d-ada1f86443bb None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.266073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d174349-aaf6-4734-b03d-ada1f86443bb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.266073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d174349-aaf6-4734-b03d-ada1f86443bb None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.266073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 37/70] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:51.266073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d174349-aaf6-4734-b03d-ada1f86443bb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:51.333387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c0e9a-944c-41a6-ac04-93706ab211e9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:51.371302 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66ebe1-a3d4-4f82-adf8-e9ed62e91dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:51.372066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66ebe1-a3d4-4f82-adf8-e9ed62e91dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.372355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66ebe1-a3d4-4f82-adf8-e9ed62e91dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.372641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66ebe1-a3d4-4f82-adf8-e9ed62e91dc6 None admin] PATH_INFO: `/v3/users/3d8ca4abbdac4f939233a06b4533d187` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.377313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d174349-aaf6-4734-b03d-ada1f86443bb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:51.388677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66ebe1-a3d4-4f82-adf8-e9ed62e91dc6 None admin] RBAC: Authorizing `identity:get_user(user_id=3d8ca4abbdac4f939233a06b4533d187)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:51.390143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66ebe1-a3d4-4f82-adf8-e9ed62e91dc6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:51.391791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 34/71] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:51 2022] GET /identity/v3/users/3d8ca4abbdac4f939233a06b4533d187 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:51.392975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cd958-15c8-441b-8506-1d69e4dfab71 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.393263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cd958-15c8-441b-8506-1d69e4dfab71 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.393539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cd958-15c8-441b-8506-1d69e4dfab71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.411094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-436cd958-15c8-441b-8506-1d69e4dfab71 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:51.412263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-436cd958-15c8-441b-8506-1d69e4dfab71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:51.413284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce0da4-5912-439e-957c-8fe0290a7231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:51.413892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0da4-5912-439e-957c-8fe0290a7231 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.414178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0da4-5912-439e-957c-8fe0290a7231 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.414452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0da4-5912-439e-957c-8fe0290a7231 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.415123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce0da4-5912-439e-957c-8fe0290a7231 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:51.415430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce0da4-5912-439e-957c-8fe0290a7231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:51.416858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce0da4-5912-439e-957c-8fe0290a7231 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:51.425239 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 38/72] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:18:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:51.426368 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fd113-b958-4f7d-8f6a-100ddb5411d5 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.426665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fd113-b958-4f7d-8f6a-100ddb5411d5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.427039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fd113-b958-4f7d-8f6a-100ddb5411d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.445737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a77fd113-b958-4f7d-8f6a-100ddb5411d5 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:51.446937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a77fd113-b958-4f7d-8f6a-100ddb5411d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:51.484143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 35/73] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:51.485474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251bb68-9d84-4b73-ac73-10245b4bb486 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.485771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251bb68-9d84-4b73-ac73-10245b4bb486 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.486055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251bb68-9d84-4b73-ac73-10245b4bb486 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.487303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 36/74] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:51.488172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251bb68-9d84-4b73-ac73-10245b4bb486 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:51.499906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:51.506873 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:51.518020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 39/75] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:51.518892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fd113-b958-4f7d-8f6a-100ddb5411d5 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:51.595254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251bb68-9d84-4b73-ac73-10245b4bb486 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:51.616297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fd113-b958-4f7d-8f6a-100ddb5411d5 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:51.629096 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a2bea-2d29-48a7-9086-daf51efc5cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:51.629692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a2bea-2d29-48a7-9086-daf51efc5cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.629976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a2bea-2d29-48a7-9086-daf51efc5cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.630244 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a2bea-2d29-48a7-9086-daf51efc5cf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.649741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617a879-b7d1-4a2f-bda1-b5733e2fb47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:51.650379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617a879-b7d1-4a2f-bda1-b5733e2fb47d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.650652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617a879-b7d1-4a2f-bda1-b5733e2fb47d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.650922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617a879-b7d1-4a2f-bda1-b5733e2fb47d None admin] PATH_INFO: `/v3/users/3d8ca4abbdac4f939233a06b4533d187` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.652577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617a879-b7d1-4a2f-bda1-b5733e2fb47d None admin] RBAC: Authorizing `identity:get_user(user_id=3d8ca4abbdac4f939233a06b4533d187)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:51.654039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617a879-b7d1-4a2f-bda1-b5733e2fb47d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:51.659912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 40/76] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:51 2022] GET /identity/v3/users/3d8ca4abbdac4f939233a06b4533d187 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:51.659912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00db8d-0e3a-40ca-bb49-4a7aebc14c75 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.659912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00db8d-0e3a-40ca-bb49-4a7aebc14c75 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.659912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00db8d-0e3a-40ca-bb49-4a7aebc14c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.674865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cc00db8d-0e3a-40ca-bb49-4a7aebc14c75 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:51.675980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cc00db8d-0e3a-40ca-bb49-4a7aebc14c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:51.697485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a2bea-2d29-48a7-9086-daf51efc5cf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:51.699044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a2bea-2d29-48a7-9086-daf51efc5cf8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:51.708493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:51.729806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:51.741652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 41/77] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:51.742480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00db8d-0e3a-40ca-bb49-4a7aebc14c75 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:51.799510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 37/78] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:18:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:18:51.800381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a2bea-2d29-48a7-9086-daf51efc5cf8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:51.855720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00db8d-0e3a-40ca-bb49-4a7aebc14c75 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:51.889731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e816a-ddf1-468b-945a-7bf9515efe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:51.890355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e816a-ddf1-468b-945a-7bf9515efe4d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.890635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e816a-ddf1-468b-945a-7bf9515efe4d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.890909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e816a-ddf1-468b-945a-7bf9515efe4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.898570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a2bea-2d29-48a7-9086-daf51efc5cf8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:51.900723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e816a-ddf1-468b-945a-7bf9515efe4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-803e816a-ddf1-468b-945a-7bf9515efe4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:51.905921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:51.906747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:18:51.907496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 42/79] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:51.908835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e816a-ddf1-468b-945a-7bf9515efe4d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:51.912738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:51.937563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066a937-3e5d-4a66-bc6c-0fe3771a38aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:51.938125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066a937-3e5d-4a66-bc6c-0fe3771a38aa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:51.938409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066a937-3e5d-4a66-bc6c-0fe3771a38aa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:51.938685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066a937-3e5d-4a66-bc6c-0fe3771a38aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:51.940209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066a937-3e5d-4a66-bc6c-0fe3771a38aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:51.941288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066a937-3e5d-4a66-bc6c-0fe3771a38aa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:51.944195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 38/80] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:18:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:51.945222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066a937-3e5d-4a66-bc6c-0fe3771a38aa None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:51.958512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:52.009303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e816a-ddf1-468b-945a-7bf9515efe4d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:52.042083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3143e-4e71-4048-b6c2-8f718560124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:52.043175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3143e-4e71-4048-b6c2-8f718560124e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:52.043455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3143e-4e71-4048-b6c2-8f718560124e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:52.043725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3143e-4e71-4048-b6c2-8f718560124e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:52.049301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066a937-3e5d-4a66-bc6c-0fe3771a38aa None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:52.083636 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96842920-3608-4016-b774-aa0949a22946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:52.084284 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96842920-3608-4016-b774-aa0949a22946 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:52.084567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96842920-3608-4016-b774-aa0949a22946 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:52.084851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96842920-3608-4016-b774-aa0949a22946 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:52.094670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96842920-3608-4016-b774-aa0949a22946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-96842920-3608-4016-b774-aa0949a22946 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:52.099334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:52.100294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:18:52.101036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 39/81] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:18:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:52.101697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96842920-3608-4016-b774-aa0949a22946 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:52.112938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3143e-4e71-4048-b6c2-8f718560124e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:52.114414 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3143e-4e71-4048-b6c2-8f718560124e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:52.139980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:52.163122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:52.203268 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96842920-3608-4016-b774-aa0949a22946 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:52.213943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 43/82] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:18:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:18:52.214893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3143e-4e71-4048-b6c2-8f718560124e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:52.237987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de33d68-b5fb-4174-8e5a-5f9756c5cbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:52.238613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de33d68-b5fb-4174-8e5a-5f9756c5cbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:52.238951 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de33d68-b5fb-4174-8e5a-5f9756c5cbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:52.239237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de33d68-b5fb-4174-8e5a-5f9756c5cbd6 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:52.240403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de33d68-b5fb-4174-8e5a-5f9756c5cbd6 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:52.241852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de33d68-b5fb-4174-8e5a-5f9756c5cbd6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:52.243470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 40/83] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:18:52 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:52.244480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de33d68-b5fb-4174-8e5a-5f9756c5cbd6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:52.314531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3143e-4e71-4048-b6c2-8f718560124e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:52.342266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de33d68-b5fb-4174-8e5a-5f9756c5cbd6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:52.344581 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:52.349497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d425e-ddea-434b-bfb3-ee152b7940d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:52.350107 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d425e-ddea-434b-bfb3-ee152b7940d8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:52.350389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d425e-ddea-434b-bfb3-ee152b7940d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:52.350664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d425e-ddea-434b-bfb3-ee152b7940d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:52.351388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d425e-ddea-434b-bfb3-ee152b7940d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:52.351696 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d425e-ddea-434b-bfb3-ee152b7940d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:52.353122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d425e-ddea-434b-bfb3-ee152b7940d8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:52.361711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 44/84] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:52.362336 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d425e-ddea-434b-bfb3-ee152b7940d8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:52.377626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:52.381132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a518adc-1766-4a1b-8c88-7374d76023b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:52.381910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a518adc-1766-4a1b-8c88-7374d76023b1 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:52.385775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a518adc-1766-4a1b-8c88-7374d76023b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:52.385775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a518adc-1766-4a1b-8c88-7374d76023b1 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:52.385775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a518adc-1766-4a1b-8c88-7374d76023b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:52.385775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a518adc-1766-4a1b-8c88-7374d76023b1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:52.457031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 41/85] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:18:52 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:18:52.457792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a518adc-1766-4a1b-8c88-7374d76023b1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:52.471245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d425e-ddea-434b-bfb3-ee152b7940d8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:52.517442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2d911-f430-4a3d-9f75-3b0e75c54716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:52.518122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2d911-f430-4a3d-9f75-3b0e75c54716 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:52.518453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2d911-f430-4a3d-9f75-3b0e75c54716 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:52.518735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2d911-f430-4a3d-9f75-3b0e75c54716 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:52.519533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2d911-f430-4a3d-9f75-3b0e75c54716 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:52.520170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2d911-f430-4a3d-9f75-3b0e75c54716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:52.521648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2d911-f430-4a3d-9f75-3b0e75c54716 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:52.532605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 45/86] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:18:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:52.533340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2d911-f430-4a3d-9f75-3b0e75c54716 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:52.576851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a518adc-1766-4a1b-8c88-7374d76023b1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:52.615645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8241e1-6815-485e-a2fe-c6b9d49557f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:52.616399 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8241e1-6815-485e-a2fe-c6b9d49557f6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:52.616588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8241e1-6815-485e-a2fe-c6b9d49557f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:52.617507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8241e1-6815-485e-a2fe-c6b9d49557f6 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:52.618022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8241e1-6815-485e-a2fe-c6b9d49557f6 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:52.618277 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:52.620503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8241e1-6815-485e-a2fe-c6b9d49557f6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:52.621624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 42/87] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:18:52 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:52.622536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8241e1-6815-485e-a2fe-c6b9d49557f6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:52.654251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2d911-f430-4a3d-9f75-3b0e75c54716 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:52.662601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:52.741113 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba1f7d-1a56-43e5-b351-026d086516d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:52.741767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba1f7d-1a56-43e5-b351-026d086516d9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:52.742128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba1f7d-1a56-43e5-b351-026d086516d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:52.742481 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba1f7d-1a56-43e5-b351-026d086516d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:52.743228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba1f7d-1a56-43e5-b351-026d086516d9 None admin] RBAC: Adding query filter params (role.id=87ab4761555e43419d0f23a0f31114f3, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user.id=3d8ca4abbdac4f939233a06b4533d187) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:52.743530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8241e1-6815-485e-a2fe-c6b9d49557f6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:52.743627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba1f7d-1a56-43e5-b351-026d086516d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:52.745369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba1f7d-1a56-43e5-b351-026d086516d9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:52.754050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 46/88] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:18:52 2022] GET /identity/v3/role_assignments?user.id=3d8ca4abbdac4f939233a06b4533d187&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:52.754900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba1f7d-1a56-43e5-b351-026d086516d9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:52.791195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fa73b-c55d-48a2-850f-719c7eebf15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:52.791861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fa73b-c55d-48a2-850f-719c7eebf15c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:52.792342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fa73b-c55d-48a2-850f-719c7eebf15c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:52.792730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fa73b-c55d-48a2-850f-719c7eebf15c None admin] PATH_INFO: `/v3/users/620b714c359944a6b1d8649f781a6815` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:52.794322 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fa73b-c55d-48a2-850f-719c7eebf15c None admin] RBAC: Authorizing `identity:get_user(user_id=620b714c359944a6b1d8649f781a6815)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:52.795964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fa73b-c55d-48a2-850f-719c7eebf15c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:52.797981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 43/89] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:52 2022] GET /identity/v3/users/620b714c359944a6b1d8649f781a6815 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:52.799371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fa73b-c55d-48a2-850f-719c7eebf15c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:52.931071 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba1f7d-1a56-43e5-b351-026d086516d9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:52.932594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:52.949968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fa73b-c55d-48a2-850f-719c7eebf15c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:52.996274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375568e-7c8b-4cac-a001-7339f15f5b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:52.997790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375568e-7c8b-4cac-a001-7339f15f5b24 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:52.998372 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375568e-7c8b-4cac-a001-7339f15f5b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:52.998821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375568e-7c8b-4cac-a001-7339f15f5b24 None admin] PATH_INFO: `/v3/users/3d8ca4abbdac4f939233a06b4533d187` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:53.000225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375568e-7c8b-4cac-a001-7339f15f5b24 None admin] RBAC: Authorizing `identity:get_user(user_id=3d8ca4abbdac4f939233a06b4533d187)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:53.001876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375568e-7c8b-4cac-a001-7339f15f5b24 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:53.002177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d00517-61f6-4da3-b668-d328413cea75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:53.002923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d00517-61f6-4da3-b668-d328413cea75 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:53.003274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d00517-61f6-4da3-b668-d328413cea75 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:53.003617 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d00517-61f6-4da3-b668-d328413cea75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:53.004503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d00517-61f6-4da3-b668-d328413cea75 None admin] RBAC: Adding query filter params (role.id=b69a8899fce14a0d839745bb76b0f441, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user.id=3d8ca4abbdac4f939233a06b4533d187) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:53.004678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 47/90] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:52 2022] GET /identity/v3/users/3d8ca4abbdac4f939233a06b4533d187 => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:53.004932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d00517-61f6-4da3-b668-d328413cea75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:53.005770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375568e-7c8b-4cac-a001-7339f15f5b24 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:53.010882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d00517-61f6-4da3-b668-d328413cea75 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:53.019059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:53.021797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 44/91] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:18:52 2022] GET /identity/v3/role_assignments?user.id=3d8ca4abbdac4f939233a06b4533d187&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=b69a8899fce14a0d839745bb76b0f441 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:53.025721 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d00517-61f6-4da3-b668-d328413cea75 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:53.195789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375568e-7c8b-4cac-a001-7339f15f5b24 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:53.236644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e57943-c9a0-4688-b0cf-9638c6637689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:53.238124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e57943-c9a0-4688-b0cf-9638c6637689 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:53.238716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e57943-c9a0-4688-b0cf-9638c6637689 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:53.239160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e57943-c9a0-4688-b0cf-9638c6637689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:53.265662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d00517-61f6-4da3-b668-d328413cea75 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:53.328320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e57943-c9a0-4688-b0cf-9638c6637689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:53.330453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e57943-c9a0-4688-b0cf-9638c6637689 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:53.340525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:53.356755 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f8096-a141-422f-b9af-db32d5c724e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:53.361149 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f8096-a141-422f-b9af-db32d5c724e4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:53.361149 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f8096-a141-422f-b9af-db32d5c724e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:53.361149 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f8096-a141-422f-b9af-db32d5c724e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:53.429010 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:53.446813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 48/92] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:18:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:18:53.452947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e57943-c9a0-4688-b0cf-9638c6637689 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:53.523277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f8096-a141-422f-b9af-db32d5c724e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:53.524686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f8096-a141-422f-b9af-db32d5c724e4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:53.602663 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e57943-c9a0-4688-b0cf-9638c6637689 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:53.611323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:53.652671 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea7385-9a31-48e3-b64e-ac9da5707d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:53.653327 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7385-9a31-48e3-b64e-ac9da5707d76 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:53.653597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7385-9a31-48e3-b64e-ac9da5707d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:53.653932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7385-9a31-48e3-b64e-ac9da5707d76 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:53.658072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea7385-9a31-48e3-b64e-ac9da5707d76 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:53.659623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea7385-9a31-48e3-b64e-ac9da5707d76 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:53.665629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 49/93] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:18:53 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:53.670339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea7385-9a31-48e3-b64e-ac9da5707d76 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:53.779392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 45/94] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:18:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 754 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:18:53.784745 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f8096-a141-422f-b9af-db32d5c724e4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:53.902171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea7385-9a31-48e3-b64e-ac9da5707d76 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:53.915563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:53.986890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f8096-a141-422f-b9af-db32d5c724e4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:53.991521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d525692-e98e-486d-8c97-2b53a1735b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:53.992189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d525692-e98e-486d-8c97-2b53a1735b1f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:53.992475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d525692-e98e-486d-8c97-2b53a1735b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:53.992754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d525692-e98e-486d-8c97-2b53a1735b1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:53.993616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d525692-e98e-486d-8c97-2b53a1735b1f None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user.id=620b714c359944a6b1d8649f781a6815) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:53.993932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d525692-e98e-486d-8c97-2b53a1735b1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:53.996339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d525692-e98e-486d-8c97-2b53a1735b1f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:54.008401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 50/95] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:18:53 2022] GET /identity/v3/role_assignments?user.id=620b714c359944a6b1d8649f781a6815&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=108664305d2546949f7327b65d4cb667 => generated 259 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:54.048654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b83f2-863a-4de5-a967-ea095903c425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:54.049535 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b83f2-863a-4de5-a967-ea095903c425 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:54.049966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b83f2-863a-4de5-a967-ea095903c425 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:54.050389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b83f2-863a-4de5-a967-ea095903c425 None admin] PATH_INFO: `/v3/projects/26b9cf71b4654229b9cea727c2e020a1` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:54.057093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b83f2-863a-4de5-a967-ea095903c425 None admin] RBAC: Authorizing `identity:get_project(project_id=26b9cf71b4654229b9cea727c2e020a1)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:54.059449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b83f2-863a-4de5-a967-ea095903c425 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:54.061455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 46/96] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:18:53 2022] GET /identity/v3/projects/26b9cf71b4654229b9cea727c2e020a1 => generated 312 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:54.068107 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d525692-e98e-486d-8c97-2b53a1735b1f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:54.135339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:54.216830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d525692-e98e-486d-8c97-2b53a1735b1f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:54.269353 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4cb6d6-23aa-419f-9d78-614685fce5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:54.269984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cb6d6-23aa-419f-9d78-614685fce5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:54.270267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cb6d6-23aa-419f-9d78-614685fce5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:54.270544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cb6d6-23aa-419f-9d78-614685fce5f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:54.271283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4cb6d6-23aa-419f-9d78-614685fce5f9 None admin] RBAC: Adding query filter params (role.id=87ab4761555e43419d0f23a0f31114f3, scope.project.id=26b9cf71b4654229b9cea727c2e020a1, user.id=3d8ca4abbdac4f939233a06b4533d187) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:54.271598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4cb6d6-23aa-419f-9d78-614685fce5f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:54.273078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4cb6d6-23aa-419f-9d78-614685fce5f9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:54.279382 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 51/97] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:18:54 2022] GET /identity/v3/role_assignments?user.id=3d8ca4abbdac4f939233a06b4533d187&scope.project.id=26b9cf71b4654229b9cea727c2e020a1&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:55.161287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25820f4a-b410-43e0-92c8-256cf7f82e6c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:55.161791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25820f4a-b410-43e0-92c8-256cf7f82e6c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:55.161864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25820f4a-b410-43e0-92c8-256cf7f82e6c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:55.163350 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 47/98] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:55.197115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868e68a-2929-46d3-8b4b-9b4bbd5ac08c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:55.197585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868e68a-2929-46d3-8b4b-9b4bbd5ac08c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:55.197585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868e68a-2929-46d3-8b4b-9b4bbd5ac08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:55.202209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec842ad-f14e-4d33-b089-0939c5e2d8ce None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:55.202496 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec842ad-f14e-4d33-b089-0939c5e2d8ce None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:55.202777 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec842ad-f14e-4d33-b089-0939c5e2d8ce None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:55.204304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 48/99] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:55.213977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40647548-a359-4a77-8e79-d78855e4da9e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:55.214270 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40647548-a359-4a77-8e79-d78855e4da9e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:55.214616 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40647548-a359-4a77-8e79-d78855e4da9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:55.238909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9868e68a-2929-46d3-8b4b-9b4bbd5ac08c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:55.240146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9868e68a-2929-46d3-8b4b-9b4bbd5ac08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:55.246983 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-40647548-a359-4a77-8e79-d78855e4da9e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:55.248187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-40647548-a359-4a77-8e79-d78855e4da9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:55.369933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 49/100] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:18:55.372190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 52/101] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:55.395622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40647548-a359-4a77-8e79-d78855e4da9e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:55.404650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868e68a-2929-46d3-8b4b-9b4bbd5ac08c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:55.468464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:55.544376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:55.624292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40647548-a359-4a77-8e79-d78855e4da9e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:55.660753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868e68a-2929-46d3-8b4b-9b4bbd5ac08c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:55.702690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f72c38-0c76-4b52-9400-f8c88f74c6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:55.703440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f72c38-0c76-4b52-9400-f8c88f74c6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:55.703726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f72c38-0c76-4b52-9400-f8c88f74c6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:55.704006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f72c38-0c76-4b52-9400-f8c88f74c6c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:55.723827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef5f58-92ff-46e1-9722-78a862a6e206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:55.724427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef5f58-92ff-46e1-9722-78a862a6e206 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:55.724710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef5f58-92ff-46e1-9722-78a862a6e206 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:55.724986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef5f58-92ff-46e1-9722-78a862a6e206 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:55.740330 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f72c38-0c76-4b52-9400-f8c88f74c6c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:55.745071 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef5f58-92ff-46e1-9722-78a862a6e206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-99f72c38-0c76-4b52-9400-f8c88f74c6c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:55.745425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:55.746472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:18:55.747623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 50/102] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:18:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-19ef5f58-92ff-46e1-9722-78a862a6e206 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:55.749937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:55.750789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:18:55.752559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 53/103] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:18:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:55.756844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f72c38-0c76-4b52-9400-f8c88f74c6c7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:55.759495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef5f58-92ff-46e1-9722-78a862a6e206 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:55.912826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef5f58-92ff-46e1-9722-78a862a6e206 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:55.916350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:55.990400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a9ee0-7c73-4c8e-828a-d5080db66a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:55.990938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a9ee0-7c73-4c8e-828a-d5080db66a16 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:55.991222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a9ee0-7c73-4c8e-828a-d5080db66a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:55.991497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a9ee0-7c73-4c8e-828a-d5080db66a16 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:55.992252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a9ee0-7c73-4c8e-828a-d5080db66a16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:55.992786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a9ee0-7c73-4c8e-828a-d5080db66a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:55.994565 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a9ee0-7c73-4c8e-828a-d5080db66a16 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:56.004560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 54/104] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:18:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:56.006245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c0af4-2f23-44df-8900-c0104c5812fc None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.006679 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c0af4-2f23-44df-8900-c0104c5812fc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.007113 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c0af4-2f23-44df-8900-c0104c5812fc None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.008508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 55/105] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:56.015192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f72c38-0c76-4b52-9400-f8c88f74c6c7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:56.016890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c0af4-2f23-44df-8900-c0104c5812fc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:56.037231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:56.055693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5a967-f0aa-45c6-b070-ff1661987afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:56.056336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5a967-f0aa-45c6-b070-ff1661987afc None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.056625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5a967-f0aa-45c6-b070-ff1661987afc None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.056922 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5a967-f0aa-45c6-b070-ff1661987afc None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.057691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5a967-f0aa-45c6-b070-ff1661987afc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:56.058000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5a967-f0aa-45c6-b070-ff1661987afc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:56.059581 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5a967-f0aa-45c6-b070-ff1661987afc None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:56.070356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 51/106] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:18:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:56.071821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8fa41-9853-4094-821e-3b0304bc2bb8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.076286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8fa41-9853-4094-821e-3b0304bc2bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.076826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8fa41-9853-4094-821e-3b0304bc2bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.096766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3ac8fa41-9853-4094-821e-3b0304bc2bb8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:56.098280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3ac8fa41-9853-4094-821e-3b0304bc2bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:56.145858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c0af4-2f23-44df-8900-c0104c5812fc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:56.180090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 52/107] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:56.187102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b75e52-bf3c-46ad-a9bc-a8e69fafe899 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.187666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b75e52-bf3c-46ad-a9bc-a8e69fafe899 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.190287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2705c8-e155-4cac-a01a-01639c2c0699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:56.190287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2705c8-e155-4cac-a01a-01639c2c0699 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.190287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2705c8-e155-4cac-a01a-01639c2c0699 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.190287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2705c8-e155-4cac-a01a-01639c2c0699 None admin] PATH_INFO: `/v3/users/3d8ca4abbdac4f939233a06b4533d187` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.191023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b75e52-bf3c-46ad-a9bc-a8e69fafe899 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.191464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2705c8-e155-4cac-a01a-01639c2c0699 None admin] RBAC: Authorizing `identity:get_user(user_id=3d8ca4abbdac4f939233a06b4533d187)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:56.192742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 53/108] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:56.193162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2705c8-e155-4cac-a01a-01639c2c0699 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:56.193863 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b75e52-bf3c-46ad-a9bc-a8e69fafe899 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:56.194982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 56/109] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:56 2022] GET /identity/v3/users/3d8ca4abbdac4f939233a06b4533d187 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:56.200557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2705c8-e155-4cac-a01a-01639c2c0699 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:56.237468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:56.324042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2705c8-e155-4cac-a01a-01639c2c0699 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:56.329244 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b75e52-bf3c-46ad-a9bc-a8e69fafe899 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:56.343526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:56.371422 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3214d6c-8027-4841-bc6f-4dda7c823a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:56.372452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3214d6c-8027-4841-bc6f-4dda7c823a83 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.372870 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3214d6c-8027-4841-bc6f-4dda7c823a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.373278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3214d6c-8027-4841-bc6f-4dda7c823a83 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.373436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de66ba6-021b-43f0-a237-8b8b058338ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:56.374316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de66ba6-021b-43f0-a237-8b8b058338ea None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.374553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3214d6c-8027-4841-bc6f-4dda7c823a83 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:56.374889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de66ba6-021b-43f0-a237-8b8b058338ea None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.375306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de66ba6-021b-43f0-a237-8b8b058338ea None admin] PATH_INFO: `/v3/users/fcb20c7bffdd41e8a68646bd5c99d3d4` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.375997 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3214d6c-8027-4841-bc6f-4dda7c823a83 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:56.380060 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 57/110] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:18:56 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:56.380060 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b62fc6-3f08-4d9e-910b-eb9c9704534c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.380060 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b62fc6-3f08-4d9e-910b-eb9c9704534c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.380060 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b62fc6-3f08-4d9e-910b-eb9c9704534c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.396252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de66ba6-021b-43f0-a237-8b8b058338ea None admin] RBAC: Authorizing `identity:get_user(user_id=fcb20c7bffdd41e8a68646bd5c99d3d4)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:56.397751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de66ba6-021b-43f0-a237-8b8b058338ea None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:56.399849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 54/111] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:56 2022] GET /identity/v3/users/fcb20c7bffdd41e8a68646bd5c99d3d4 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:56.403643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c6b62fc6-3f08-4d9e-910b-eb9c9704534c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:56.404532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de66ba6-021b-43f0-a237-8b8b058338ea None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:56.404954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c6b62fc6-3f08-4d9e-910b-eb9c9704534c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:56.489335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 58/112] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:56.492190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b62fc6-3f08-4d9e-910b-eb9c9704534c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:56.528280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:56.536478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de66ba6-021b-43f0-a237-8b8b058338ea None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:56.578477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ba882-0673-4488-bc95-4eb0dcc1179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:56.579038 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba882-0673-4488-bc95-4eb0dcc1179d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.579322 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba882-0673-4488-bc95-4eb0dcc1179d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.579699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba882-0673-4488-bc95-4eb0dcc1179d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.581019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ba882-0673-4488-bc95-4eb0dcc1179d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:56.582576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ba882-0673-4488-bc95-4eb0dcc1179d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:56.584414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 55/113] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:18:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:56.585253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ba882-0673-4488-bc95-4eb0dcc1179d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:56.626345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b62fc6-3f08-4d9e-910b-eb9c9704534c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:56.636751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:56.666770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68b220-a90d-4b4f-81a3-60933fedc5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:56.669429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68b220-a90d-4b4f-81a3-60933fedc5e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.669721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68b220-a90d-4b4f-81a3-60933fedc5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.670003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68b220-a90d-4b4f-81a3-60933fedc5e7 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/users/3d8ca4abbdac4f939233a06b4533d187/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.672418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68b220-a90d-4b4f-81a3-60933fedc5e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user_id=3d8ca4abbdac4f939233a06b4533d187, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:56.673879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68b220-a90d-4b4f-81a3-60933fedc5e7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:56.712801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ba882-0673-4488-bc95-4eb0dcc1179d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:56.740057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 59/114] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:18:56 2022] PUT /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/users/3d8ca4abbdac4f939233a06b4533d187/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:18:56.741872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68b220-a90d-4b4f-81a3-60933fedc5e7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:56.757073 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d46e2-89a0-4d8c-8566-3106fac0a9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:56.757759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d46e2-89a0-4d8c-8566-3106fac0a9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.758041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d46e2-89a0-4d8c-8566-3106fac0a9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.758316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d46e2-89a0-4d8c-8566-3106fac0a9f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.852533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:56.873662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d46e2-89a0-4d8c-8566-3106fac0a9f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:56.876363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d46e2-89a0-4d8c-8566-3106fac0a9f6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:56.884882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68b220-a90d-4b4f-81a3-60933fedc5e7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:56.925946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dcb11-4f22-49d6-b528-ddf546092a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:56.926779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dcb11-4f22-49d6-b528-ddf546092a9e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:56.927155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dcb11-4f22-49d6-b528-ddf546092a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:56.927503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dcb11-4f22-49d6-b528-ddf546092a9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:56.930837 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:56.943342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713dcb11-4f22-49d6-b528-ddf546092a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-713dcb11-4f22-49d6-b528-ddf546092a9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:56.948431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:56.949493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:18:56.950362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:18:56.951915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 60/115] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:18:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:56.952544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dcb11-4f22-49d6-b528-ddf546092a9e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:57.034626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 56/116] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:18:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:18:57.036001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf13cc-2472-4411-a294-f655777a1e88 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.036379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf13cc-2472-4411-a294-f655777a1e88 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.036726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf13cc-2472-4411-a294-f655777a1e88 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.039148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 57/117] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:57.039946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf13cc-2472-4411-a294-f655777a1e88 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:57.078192 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dcb11-4f22-49d6-b528-ddf546092a9e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:57.115697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3d06e-b1f5-4f9f-93bd-fa3615df9550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:57.116739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3d06e-b1f5-4f9f-93bd-fa3615df9550 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.117282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3d06e-b1f5-4f9f-93bd-fa3615df9550 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.117780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3d06e-b1f5-4f9f-93bd-fa3615df9550 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.118636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3d06e-b1f5-4f9f-93bd-fa3615df9550 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:57.120678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3d06e-b1f5-4f9f-93bd-fa3615df9550 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:57.132604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:57.159333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf13cc-2472-4411-a294-f655777a1e88 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:57.194339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e7e2e-3502-4bad-977b-9d61839fad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:57.195001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e7e2e-3502-4bad-977b-9d61839fad60 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.195248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e7e2e-3502-4bad-977b-9d61839fad60 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.195454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e7e2e-3502-4bad-977b-9d61839fad60 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.196146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e7e2e-3502-4bad-977b-9d61839fad60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:57.196564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e7e2e-3502-4bad-977b-9d61839fad60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:57.197838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e7e2e-3502-4bad-977b-9d61839fad60 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:57.201187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:57.207551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 61/118] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:18:56 2022] POST /identity/v3/projects => generated 302 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:18:57.208046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 58/119] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:18:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:57.208699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e7e2e-3502-4bad-977b-9d61839fad60 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:57.210498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1789-533e-43d6-b8aa-78ccbf30e5c8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.210789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1789-533e-43d6-b8aa-78ccbf30e5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.211070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1789-533e-43d6-b8aa-78ccbf30e5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.242226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-55fe1789-533e-43d6-b8aa-78ccbf30e5c8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:57.243825 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-55fe1789-533e-43d6-b8aa-78ccbf30e5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:57.333557 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e7e2e-3502-4bad-977b-9d61839fad60 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:57.340479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 62/120] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:57.341766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d47151-56e7-4df9-bb89-2a1c63982bea None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.342151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d47151-56e7-4df9-bb89-2a1c63982bea None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.342432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d47151-56e7-4df9-bb89-2a1c63982bea None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.344120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 63/121] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:57.344958 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d47151-56e7-4df9-bb89-2a1c63982bea None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:57.393063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f78d36-6613-487f-b69e-5350636916f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:57.393623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f78d36-6613-487f-b69e-5350636916f4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.393906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f78d36-6613-487f-b69e-5350636916f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.394180 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f78d36-6613-487f-b69e-5350636916f4 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.395328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f78d36-6613-487f-b69e-5350636916f4 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:57.396834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f78d36-6613-487f-b69e-5350636916f4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:57.398498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 59/122] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:18:57 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:57.400495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b75c-ecac-41c5-910f-216e714aacd2 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.400786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b75c-ecac-41c5-910f-216e714aacd2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.401064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b75c-ecac-41c5-910f-216e714aacd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.421101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-def9b75c-ecac-41c5-910f-216e714aacd2 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:57.422541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-def9b75c-ecac-41c5-910f-216e714aacd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:57.482074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d47151-56e7-4df9-bb89-2a1c63982bea None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:57.494602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:57.503612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 60/123] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:18:57.504501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9b75c-ecac-41c5-910f-216e714aacd2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:57.514758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:57.521688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d08168-922e-49d6-b401-090cc867bb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:57.522362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d08168-922e-49d6-b401-090cc867bb18 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.522701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d08168-922e-49d6-b401-090cc867bb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.522979 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d08168-922e-49d6-b401-090cc867bb18 None admin] PATH_INFO: `/v3/users/3d8ca4abbdac4f939233a06b4533d187` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.524203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d08168-922e-49d6-b401-090cc867bb18 None admin] RBAC: Authorizing `identity:get_user(user_id=3d8ca4abbdac4f939233a06b4533d187)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:57.525766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d08168-922e-49d6-b401-090cc867bb18 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:57.527956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 64/124] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:57 2022] GET /identity/v3/users/3d8ca4abbdac4f939233a06b4533d187 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:57.528892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d08168-922e-49d6-b401-090cc867bb18 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:57.624407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9b75c-ecac-41c5-910f-216e714aacd2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:57.662951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d08168-922e-49d6-b401-090cc867bb18 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:57.664454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f19f5-7e12-4b68-83c3-844b6d37678f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:57.665146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f19f5-7e12-4b68-83c3-844b6d37678f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.665515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f19f5-7e12-4b68-83c3-844b6d37678f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.665703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f19f5-7e12-4b68-83c3-844b6d37678f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.682189 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f19f5-7e12-4b68-83c3-844b6d37678f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-9f4f19f5-7e12-4b68-83c3-844b6d37678f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:57.687469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:57.688374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:18:57.689157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 61/125] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:18:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:57.689767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f19f5-7e12-4b68-83c3-844b6d37678f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:57.719113 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b333a-cfe9-468b-b34a-3780f112e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:57.719903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b333a-cfe9-468b-b34a-3780f112e98c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.721409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b333a-cfe9-468b-b34a-3780f112e98c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.721531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b333a-cfe9-468b-b34a-3780f112e98c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.722313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b333a-cfe9-468b-b34a-3780f112e98c None admin] RBAC: Adding query filter params (role.id=0f9e8507916f478c80987f7c7fd3c3e9, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user.id=fcb20c7bffdd41e8a68646bd5c99d3d4) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:57.722634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b333a-cfe9-468b-b34a-3780f112e98c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:57.724115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b333a-cfe9-468b-b34a-3780f112e98c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:57.730784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 65/126] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:18:57 2022] GET /identity/v3/role_assignments?user.id=fcb20c7bffdd41e8a68646bd5c99d3d4&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=0f9e8507916f478c80987f7c7fd3c3e9 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:57.731623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b333a-cfe9-468b-b34a-3780f112e98c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:57.772634 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:57.821690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:57.861965 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f19f5-7e12-4b68-83c3-844b6d37678f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:57.914244 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b333a-cfe9-468b-b34a-3780f112e98c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:57.933698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a58ba8-945b-43db-b92f-5ab163c87336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:57.936715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a58ba8-945b-43db-b92f-5ab163c87336 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.937205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a58ba8-945b-43db-b92f-5ab163c87336 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.940041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a58ba8-945b-43db-b92f-5ab163c87336 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.962706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a58ba8-945b-43db-b92f-5ab163c87336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-04a58ba8-945b-43db-b92f-5ab163c87336 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:57.970172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:57.971103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:18:57.971851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:18:57.974895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 62/127] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:18:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:57.975902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a58ba8-945b-43db-b92f-5ab163c87336 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:57.986575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9797fd-f9e8-4870-a066-f164239f0b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:57.987543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9797fd-f9e8-4870-a066-f164239f0b77 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:57.988089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9797fd-f9e8-4870-a066-f164239f0b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:57.988608 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9797fd-f9e8-4870-a066-f164239f0b77 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:57.990268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9797fd-f9e8-4870-a066-f164239f0b77 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:57.992162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9797fd-f9e8-4870-a066-f164239f0b77 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:58.000196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 66/128] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:18:57 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:58.001325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9797fd-f9e8-4870-a066-f164239f0b77 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:58.165367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a58ba8-945b-43db-b92f-5ab163c87336 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:58.185990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:58.216960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9797fd-f9e8-4870-a066-f164239f0b77 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:58.229377 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aac5e-f725-4ca6-a5b2-cd03c96ef1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:58.230526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aac5e-f725-4ca6-a5b2-cd03c96ef1de None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:58.230811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aac5e-f725-4ca6-a5b2-cd03c96ef1de None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:58.231080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aac5e-f725-4ca6-a5b2-cd03c96ef1de None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:58.231750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aac5e-f725-4ca6-a5b2-cd03c96ef1de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:58.232075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aac5e-f725-4ca6-a5b2-cd03c96ef1de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:58.233720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aac5e-f725-4ca6-a5b2-cd03c96ef1de None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:58.246703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 63/129] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:18:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:58.247496 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aac5e-f725-4ca6-a5b2-cd03c96ef1de None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:58.258185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:58.279370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c5f8e-d2e8-4709-a1db-1c4677690a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:58.279984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c5f8e-d2e8-4709-a1db-1c4677690a8e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:58.280299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c5f8e-d2e8-4709-a1db-1c4677690a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:58.280564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c5f8e-d2e8-4709-a1db-1c4677690a8e None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:58.281236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c5f8e-d2e8-4709-a1db-1c4677690a8e None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:58.281545 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c5f8e-d2e8-4709-a1db-1c4677690a8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:58.282922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c5f8e-d2e8-4709-a1db-1c4677690a8e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:58.299943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 67/130] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:18:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:58.300804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c5f8e-d2e8-4709-a1db-1c4677690a8e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:58.450246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c5f8e-d2e8-4709-a1db-1c4677690a8e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:58.450413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aac5e-f725-4ca6-a5b2-cd03c96ef1de None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:58.501136 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82670845-60b6-431f-8db8-38f8e31766b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:58.501786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82670845-60b6-431f-8db8-38f8e31766b5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:58.502066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82670845-60b6-431f-8db8-38f8e31766b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:58.502360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82670845-60b6-431f-8db8-38f8e31766b5 None admin] PATH_INFO: `/v3/users/620b714c359944a6b1d8649f781a6815` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:58.503638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82670845-60b6-431f-8db8-38f8e31766b5 None admin] RBAC: Authorizing `identity:get_user(user_id=620b714c359944a6b1d8649f781a6815)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:58.505126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82670845-60b6-431f-8db8-38f8e31766b5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:58.506704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 68/131] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:58 2022] GET /identity/v3/users/620b714c359944a6b1d8649f781a6815 => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:58.510047 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82670845-60b6-431f-8db8-38f8e31766b5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:58.514075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f5df1-ef5a-4ff9-9b4e-bdf8c3a03dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:58.514750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f5df1-ef5a-4ff9-9b4e-bdf8c3a03dba None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:58.515032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f5df1-ef5a-4ff9-9b4e-bdf8c3a03dba None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:58.515310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f5df1-ef5a-4ff9-9b4e-bdf8c3a03dba None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/users/3d8ca4abbdac4f939233a06b4533d187/roles/b69a8899fce14a0d839745bb76b0f441` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:58.520253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f5df1-ef5a-4ff9-9b4e-bdf8c3a03dba None admin] RBAC: Authorizing `identity:create_grant(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user_id=3d8ca4abbdac4f939233a06b4533d187, role_id=b69a8899fce14a0d839745bb76b0f441)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:58.521707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f5df1-ef5a-4ff9-9b4e-bdf8c3a03dba None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:58.580385 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:58.634430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 64/132] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:18:58 2022] PUT /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/users/3d8ca4abbdac4f939233a06b4533d187/roles/b69a8899fce14a0d839745bb76b0f441 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:18:58.635356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f5df1-ef5a-4ff9-9b4e-bdf8c3a03dba None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:58.642275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82670845-60b6-431f-8db8-38f8e31766b5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:58.703614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde2fe4-1598-4ef9-8159-9bf159788e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:58.704647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde2fe4-1598-4ef9-8159-9bf159788e10 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:58.705155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde2fe4-1598-4ef9-8159-9bf159788e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:58.705661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde2fe4-1598-4ef9-8159-9bf159788e10 None admin] PATH_INFO: `/v3/users/3d8ca4abbdac4f939233a06b4533d187` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:58.707681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde2fe4-1598-4ef9-8159-9bf159788e10 None admin] RBAC: Authorizing `identity:get_user(user_id=3d8ca4abbdac4f939233a06b4533d187)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:58.709198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde2fe4-1598-4ef9-8159-9bf159788e10 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:58.711404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 69/133] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:58 2022] GET /identity/v3/users/3d8ca4abbdac4f939233a06b4533d187 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:58.717369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde2fe4-1598-4ef9-8159-9bf159788e10 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:58.724701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:58.843128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde2fe4-1598-4ef9-8159-9bf159788e10 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:58.880426 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:58.902921 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f5df1-ef5a-4ff9-9b4e-bdf8c3a03dba None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:58.906190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe2a5e-2ea1-4abb-9e7a-c4f76b6de8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:58.906956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe2a5e-2ea1-4abb-9e7a-c4f76b6de8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:58.907167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe2a5e-2ea1-4abb-9e7a-c4f76b6de8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:58.907495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe2a5e-2ea1-4abb-9e7a-c4f76b6de8a4 None admin] PATH_INFO: `/v3/projects/26b9cf71b4654229b9cea727c2e020a1` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:58.910739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe2a5e-2ea1-4abb-9e7a-c4f76b6de8a4 None admin] RBAC: Authorizing `identity:get_project(project_id=26b9cf71b4654229b9cea727c2e020a1)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:58.912311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe2a5e-2ea1-4abb-9e7a-c4f76b6de8a4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:58.914499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 70/134] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:18:58 2022] GET /identity/v3/projects/26b9cf71b4654229b9cea727c2e020a1 => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:18:58.922561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe2a5e-2ea1-4abb-9e7a-c4f76b6de8a4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:58.986254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a67d5b-920f-4964-b8df-bd740cf2a72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:58.986812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a67d5b-920f-4964-b8df-bd740cf2a72b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:58.987110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a67d5b-920f-4964-b8df-bd740cf2a72b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:58.987367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a67d5b-920f-4964-b8df-bd740cf2a72b None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:58.992917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a67d5b-920f-4964-b8df-bd740cf2a72b None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:58.994761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a67d5b-920f-4964-b8df-bd740cf2a72b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:58.999138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 65/135] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:18:58 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:59.004364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a67d5b-920f-4964-b8df-bd740cf2a72b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:59.118228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe2a5e-2ea1-4abb-9e7a-c4f76b6de8a4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:59.147942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a67d5b-920f-4964-b8df-bd740cf2a72b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:59.156186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:59.184661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125c398-3f0d-44fc-b090-2bc1d5e0b92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:59.185384 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125c398-3f0d-44fc-b090-2bc1d5e0b92b None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:59.185667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125c398-3f0d-44fc-b090-2bc1d5e0b92b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:59.185944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125c398-3f0d-44fc-b090-2bc1d5e0b92b None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/users/620b714c359944a6b1d8649f781a6815/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:59.187941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125c398-3f0d-44fc-b090-2bc1d5e0b92b None admin] RBAC: Authorizing `identity:create_grant(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user_id=620b714c359944a6b1d8649f781a6815, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:59.189471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125c398-3f0d-44fc-b090-2bc1d5e0b92b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:59.198281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8033356-0697-454c-b03c-6aebf1140a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:59.200278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8033356-0697-454c-b03c-6aebf1140a5d None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:59.200614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8033356-0697-454c-b03c-6aebf1140a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:59.200944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8033356-0697-454c-b03c-6aebf1140a5d None admin] PATH_INFO: `/v3/projects/26b9cf71b4654229b9cea727c2e020a1/users/3d8ca4abbdac4f939233a06b4533d187/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:59.203019 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8033356-0697-454c-b03c-6aebf1140a5d None admin] RBAC: Authorizing `identity:create_grant(project_id=26b9cf71b4654229b9cea727c2e020a1, user_id=3d8ca4abbdac4f939233a06b4533d187, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:59.204571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8033356-0697-454c-b03c-6aebf1140a5d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:59.262815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 66/136] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:18:59 2022] PUT /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/users/620b714c359944a6b1d8649f781a6815/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:18:59.264170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b278177-8bee-4f22-a1b7-f8e1f6e13866 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:59.264504 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b278177-8bee-4f22-a1b7-f8e1f6e13866 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:59.264778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b278177-8bee-4f22-a1b7-f8e1f6e13866 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:59.267549 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 67/137] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:18:59.271720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9586d0-5dec-4c22-b456-ced850e4d07b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:59.272004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9586d0-5dec-4c22-b456-ced850e4d07b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:59.272359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9586d0-5dec-4c22-b456-ced850e4d07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:59.296119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 71/138] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:18:58 2022] PUT /identity/v3/projects/26b9cf71b4654229b9cea727c2e020a1/users/3d8ca4abbdac4f939233a06b4533d187/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:18:59.301358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6b9586d0-5dec-4c22-b456-ced850e4d07b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:18:59.303166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6b9586d0-5dec-4c22-b456-ced850e4d07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:18:59.417727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 68/139] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:18:59.451785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8033356-0697-454c-b03c-6aebf1140a5d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:59.521178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:59.581962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8033356-0697-454c-b03c-6aebf1140a5d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:59.629669 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd560a9-c00b-4022-ae27-bc9d47f96da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:59.630400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd560a9-c00b-4022-ae27-bc9d47f96da0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:59.630678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd560a9-c00b-4022-ae27-bc9d47f96da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:59.630954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd560a9-c00b-4022-ae27-bc9d47f96da0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:59.642835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd560a9-c00b-4022-ae27-bc9d47f96da0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-6cd560a9-c00b-4022-ae27-bc9d47f96da0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:18:59.652714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:18:59.653588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:18:59.654347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 72/140] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:18:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:18:59.660070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9586d0-5dec-4c22-b456-ced850e4d07b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:18:59.800353 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:18:59.806035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9586d0-5dec-4c22-b456-ced850e4d07b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:18:59.860685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b49cb-1ce0-4504-a43e-9df7739564fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:18:59.861512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b49cb-1ce0-4504-a43e-9df7739564fe None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:18:59.861732 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b49cb-1ce0-4504-a43e-9df7739564fe None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:18:59.862460 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b49cb-1ce0-4504-a43e-9df7739564fe None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:18:59.863163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b49cb-1ce0-4504-a43e-9df7739564fe None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:18:59.863720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b49cb-1ce0-4504-a43e-9df7739564fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:18:59.865570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b49cb-1ce0-4504-a43e-9df7739564fe None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:18:59.878930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 69/141] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:18:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:18:59.884739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd560a9-c00b-4022-ae27-bc9d47f96da0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:00.023790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd560a9-c00b-4022-ae27-bc9d47f96da0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:00.036912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:00.065069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d36f8b-324a-49a9-8d9b-6fcbee98e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:00.065628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d36f8b-324a-49a9-8d9b-6fcbee98e45c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:00.065907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d36f8b-324a-49a9-8d9b-6fcbee98e45c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:00.066183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d36f8b-324a-49a9-8d9b-6fcbee98e45c None admin] PATH_INFO: `/v3/users/3d8ca4abbdac4f939233a06b4533d187` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:00.067341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d36f8b-324a-49a9-8d9b-6fcbee98e45c None admin] RBAC: Authorizing `identity:get_user(user_id=3d8ca4abbdac4f939233a06b4533d187)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:00.068915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d36f8b-324a-49a9-8d9b-6fcbee98e45c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:00.070805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 73/142] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:18:59 2022] GET /identity/v3/users/3d8ca4abbdac4f939233a06b4533d187 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:00.076603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b49cb-1ce0-4504-a43e-9df7739564fe None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:00.195843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b49cb-1ce0-4504-a43e-9df7739564fe None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:00.240482 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121d082-dbb5-4f49-b307-b69fba885317 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:00.240794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121d082-dbb5-4f49-b307-b69fba885317 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:00.241033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121d082-dbb5-4f49-b307-b69fba885317 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:00.241609 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c85e54-9cf9-4e71-9477-05c148f703ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:00.242232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c85e54-9cf9-4e71-9477-05c148f703ad None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:00.242401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 74/143] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:00.242769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c85e54-9cf9-4e71-9477-05c148f703ad None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:00.243052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c85e54-9cf9-4e71-9477-05c148f703ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:00.247058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245e3a5-194d-49a1-8136-a08aa2975211 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:00.247337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245e3a5-194d-49a1-8136-a08aa2975211 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:00.247616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245e3a5-194d-49a1-8136-a08aa2975211 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:00.249042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 75/144] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:00.250917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c88d1b-06e1-4b34-8e32-21eebfd3a7d5 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:00.251195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c88d1b-06e1-4b34-8e32-21eebfd3a7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:00.251467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c88d1b-06e1-4b34-8e32-21eebfd3a7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:00.274595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f8c88d1b-06e1-4b34-8e32-21eebfd3a7d5 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:00.275907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f8c88d1b-06e1-4b34-8e32-21eebfd3a7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:00.283398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:00.353546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c85e54-9cf9-4e71-9477-05c148f703ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:00.355525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c85e54-9cf9-4e71-9477-05c148f703ad None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:00.379546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 76/145] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:00.381521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1f6d8-dd3d-4131-be46-11fe44ad3eb4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:00.381814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1f6d8-dd3d-4131-be46-11fe44ad3eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:00.382099 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1f6d8-dd3d-4131-be46-11fe44ad3eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:00.423118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2ca1f6d8-dd3d-4131-be46-11fe44ad3eb4 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:00.424413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2ca1f6d8-dd3d-4131-be46-11fe44ad3eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:00.505547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 77/146] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:00.506479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1f6d8-dd3d-4131-be46-11fe44ad3eb4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:00.554068 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:00.621456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 70/147] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:00.623448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c85e54-9cf9-4e71-9477-05c148f703ad None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:00.668159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1f6d8-dd3d-4131-be46-11fe44ad3eb4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:00.723349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c39c1-7ec5-46c3-b82b-f9b34db76c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:00.724055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c39c1-7ec5-46c3-b82b-f9b34db76c33 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:00.724347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c39c1-7ec5-46c3-b82b-f9b34db76c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:00.724676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c39c1-7ec5-46c3-b82b-f9b34db76c33 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:00.736409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c39c1-7ec5-46c3-b82b-f9b34db76c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-9b0c39c1-7ec5-46c3-b82b-f9b34db76c33 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:00.741917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:00.742749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:00.743506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 78/148] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:00.744152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c39c1-7ec5-46c3-b82b-f9b34db76c33 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:00.749207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:00.837663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c85e54-9cf9-4e71-9477-05c148f703ad None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:00.895493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:00.899039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1178824-d6d7-444f-9e07-f81f8651bf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:00.899679 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178824-d6d7-444f-9e07-f81f8651bf45 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:00.899958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178824-d6d7-444f-9e07-f81f8651bf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:00.902478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178824-d6d7-444f-9e07-f81f8651bf45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:00.905048 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1178824-d6d7-444f-9e07-f81f8651bf45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:00.906523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1178824-d6d7-444f-9e07-f81f8651bf45 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:00.908266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 71/149] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:00.908907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1178824-d6d7-444f-9e07-f81f8651bf45 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:00.913330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c39c1-7ec5-46c3-b82b-f9b34db76c33 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:00.988341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e078ec-e755-48f1-a750-d29c454d12d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:00.988989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e078ec-e755-48f1-a750-d29c454d12d7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:00.989269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e078ec-e755-48f1-a750-d29c454d12d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:00.989549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e078ec-e755-48f1-a750-d29c454d12d7 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:00.990755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e078ec-e755-48f1-a750-d29c454d12d7 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:00.992241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e078ec-e755-48f1-a750-d29c454d12d7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:00.993878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 79/150] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:00 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:00.994879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e078ec-e755-48f1-a750-d29c454d12d7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:01.114764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1178824-d6d7-444f-9e07-f81f8651bf45 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:01.137867 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e078ec-e755-48f1-a750-d29c454d12d7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:01.175314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:01.176037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df24c1-df47-4310-b800-c60125e16ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:01.180035 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df24c1-df47-4310-b800-c60125e16ae1 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:01.180035 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df24c1-df47-4310-b800-c60125e16ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:01.180035 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df24c1-df47-4310-b800-c60125e16ae1 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:01.180035 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df24c1-df47-4310-b800-c60125e16ae1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:01.180035 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df24c1-df47-4310-b800-c60125e16ae1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:01.200098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5125b5-ddee-46b0-862c-27a2097207e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:01.200781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5125b5-ddee-46b0-862c-27a2097207e6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:01.201074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5125b5-ddee-46b0-862c-27a2097207e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:01.201355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5125b5-ddee-46b0-862c-27a2097207e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:01.202113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5125b5-ddee-46b0-862c-27a2097207e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:01.202435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5125b5-ddee-46b0-862c-27a2097207e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:01.203975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5125b5-ddee-46b0-862c-27a2097207e6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:01.214505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 72/151] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:01.215432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5125b5-ddee-46b0-862c-27a2097207e6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:01.273260 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 80/152] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:19:00 2022] POST /identity/v3/users => generated 283 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:19:01.274090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df24c1-df47-4310-b800-c60125e16ae1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:01.316975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:01.336126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5125b5-ddee-46b0-862c-27a2097207e6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:01.418191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01deba93-cd53-4034-a339-086ce316da21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:01.418822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deba93-cd53-4034-a339-086ce316da21 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:01.419098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deba93-cd53-4034-a339-086ce316da21 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:01.419368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deba93-cd53-4034-a339-086ce316da21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:01.420134 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01deba93-cd53-4034-a339-086ce316da21 None admin] RBAC: Adding query filter params (role.id=87ab4761555e43419d0f23a0f31114f3, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user.id=3d8ca4abbdac4f939233a06b4533d187) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:01.420445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01deba93-cd53-4034-a339-086ce316da21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:01.421901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01deba93-cd53-4034-a339-086ce316da21 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:01.440547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 73/153] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:01 2022] GET /identity/v3/role_assignments?user.id=3d8ca4abbdac4f939233a06b4533d187&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:01.478053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df24c1-df47-4310-b800-c60125e16ae1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:01.533458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee37cc-b2b0-4969-a710-5622e58627af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:01.534108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee37cc-b2b0-4969-a710-5622e58627af None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:01.534386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee37cc-b2b0-4969-a710-5622e58627af None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:01.534659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee37cc-b2b0-4969-a710-5622e58627af None admin] PATH_INFO: `/v3/users/fcb20c7bffdd41e8a68646bd5c99d3d4` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:01.539422 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee37cc-b2b0-4969-a710-5622e58627af None admin] RBAC: Authorizing `identity:get_user(user_id=fcb20c7bffdd41e8a68646bd5c99d3d4)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:01.541049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee37cc-b2b0-4969-a710-5622e58627af None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:01.542869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 81/154] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:01 2022] GET /identity/v3/users/fcb20c7bffdd41e8a68646bd5c99d3d4 => generated 289 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:01.548865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01deba93-cd53-4034-a339-086ce316da21 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:01.610495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:01.660978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f34f8-3d04-4a91-b95f-e7ff5309f4aa None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:01.661330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f34f8-3d04-4a91-b95f-e7ff5309f4aa None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:01.661660 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f34f8-3d04-4a91-b95f-e7ff5309f4aa None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:01.663064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 82/155] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:01.664554 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01deba93-cd53-4034-a339-086ce316da21 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:01.669252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e80d0-2ccc-4f9b-8dd0-c119e9fe4a47 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:01.669527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e80d0-2ccc-4f9b-8dd0-c119e9fe4a47 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:01.669805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e80d0-2ccc-4f9b-8dd0-c119e9fe4a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:01.701096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-543e80d0-2ccc-4f9b-8dd0-c119e9fe4a47 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:01.701096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-543e80d0-2ccc-4f9b-8dd0-c119e9fe4a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:01.712763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545b97f-5f78-415e-8997-c40e921d2ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:01.713618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545b97f-5f78-415e-8997-c40e921d2ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:01.714058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545b97f-5f78-415e-8997-c40e921d2ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:01.714530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545b97f-5f78-415e-8997-c40e921d2ca2 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:01.715916 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545b97f-5f78-415e-8997-c40e921d2ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:01.717569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545b97f-5f78-415e-8997-c40e921d2ca2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:01.720145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 74/156] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:01 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:01.727755 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545b97f-5f78-415e-8997-c40e921d2ca2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:01.805423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 83/157] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:01.835206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e80d0-2ccc-4f9b-8dd0-c119e9fe4a47 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:01.858571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545b97f-5f78-415e-8997-c40e921d2ca2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:01.859863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:01.885635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:01.904898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215e122-796c-4370-b8d7-573942a70610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:01.905888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215e122-796c-4370-b8d7-573942a70610 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:01.906171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215e122-796c-4370-b8d7-573942a70610 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:01.906443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215e122-796c-4370-b8d7-573942a70610 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/users/fcb20c7bffdd41e8a68646bd5c99d3d4/roles/0f9e8507916f478c80987f7c7fd3c3e9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:01.908935 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215e122-796c-4370-b8d7-573942a70610 None admin] RBAC: Authorizing `identity:create_grant(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user_id=fcb20c7bffdd41e8a68646bd5c99d3d4, role_id=0f9e8507916f478c80987f7c7fd3c3e9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:01.910665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215e122-796c-4370-b8d7-573942a70610 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:01.968930 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e80d0-2ccc-4f9b-8dd0-c119e9fe4a47 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:02.002921 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 75/158] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:19:01 2022] PUT /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/users/fcb20c7bffdd41e8a68646bd5c99d3d4/roles/0f9e8507916f478c80987f7c7fd3c3e9 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:02.012418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb070b2-4e58-4bb8-a770-2e91fe9e67f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:02.013143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb070b2-4e58-4bb8-a770-2e91fe9e67f2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:02.013432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb070b2-4e58-4bb8-a770-2e91fe9e67f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:02.013710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb070b2-4e58-4bb8-a770-2e91fe9e67f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:02.027518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb070b2-4e58-4bb8-a770-2e91fe9e67f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-0cb070b2-4e58-4bb8-a770-2e91fe9e67f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:02.033235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:02.034167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:02.034944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 84/159] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:19:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:02.043158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215e122-796c-4370-b8d7-573942a70610 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:02.166928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b4e27-3aad-43af-8dba-b4ecd741b7b8 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:02.167245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b4e27-3aad-43af-8dba-b4ecd741b7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:02.168578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b4e27-3aad-43af-8dba-b4ecd741b7b8 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:02.170208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 85/160] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:02.170957 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215e122-796c-4370-b8d7-573942a70610 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:02.177266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e75647-8653-4d24-880d-6eadc029e61c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:02.177569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e75647-8653-4d24-880d-6eadc029e61c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:02.177831 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e75647-8653-4d24-880d-6eadc029e61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:02.201845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-36e75647-8653-4d24-880d-6eadc029e61c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:02.203081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-36e75647-8653-4d24-880d-6eadc029e61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:02.230253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a80dc-887c-49d3-a95c-f3275c8efacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:02.230787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a80dc-887c-49d3-a95c-f3275c8efacb None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:02.231063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a80dc-887c-49d3-a95c-f3275c8efacb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:02.231334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a80dc-887c-49d3-a95c-f3275c8efacb None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:02.232006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a80dc-887c-49d3-a95c-f3275c8efacb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:02.232341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a80dc-887c-49d3-a95c-f3275c8efacb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:02.233707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a80dc-887c-49d3-a95c-f3275c8efacb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:02.250656 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 76/161] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:02.256843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a80dc-887c-49d3-a95c-f3275c8efacb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:02.314524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 86/162] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:02.322937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:02.357221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e75647-8653-4d24-880d-6eadc029e61c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:02.368507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:02.397167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a80dc-887c-49d3-a95c-f3275c8efacb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:02.448949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdb807-b8ff-4aab-a0ad-d6bf51ecc6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:02.449929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdb807-b8ff-4aab-a0ad-d6bf51ecc6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:02.450347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdb807-b8ff-4aab-a0ad-d6bf51ecc6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:02.450629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdb807-b8ff-4aab-a0ad-d6bf51ecc6f4 None admin] PATH_INFO: `/v3/users/3d8ca4abbdac4f939233a06b4533d187` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:02.452120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fdb807-b8ff-4aab-a0ad-d6bf51ecc6f4 None admin] RBAC: Authorizing `identity:get_user(user_id=3d8ca4abbdac4f939233a06b4533d187)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:02.454370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fdb807-b8ff-4aab-a0ad-d6bf51ecc6f4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:02.457249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 77/163] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:02 2022] GET /identity/v3/users/3d8ca4abbdac4f939233a06b4533d187 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:02.458502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab898a4-a7c7-4a75-a559-2f1f16f8ea9e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:02.458785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab898a4-a7c7-4a75-a559-2f1f16f8ea9e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:02.459309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab898a4-a7c7-4a75-a559-2f1f16f8ea9e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:02.461331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 78/164] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:02.465469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab898a4-a7c7-4a75-a559-2f1f16f8ea9e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:02.508346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e75647-8653-4d24-880d-6eadc029e61c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:02.555690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467e933-04cf-4fe8-8a43-def2395cb24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:02.556804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467e933-04cf-4fe8-8a43-def2395cb24f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:02.557083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467e933-04cf-4fe8-8a43-def2395cb24f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:02.557614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467e933-04cf-4fe8-8a43-def2395cb24f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:02.572436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467e933-04cf-4fe8-8a43-def2395cb24f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-6467e933-04cf-4fe8-8a43-def2395cb24f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:02.579053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:02.579909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:02.580774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:02.582858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 87/165] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:02.582858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98e9c1-6b6f-42de-a69d-bc346e1ca161 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:02.582858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98e9c1-6b6f-42de-a69d-bc346e1ca161 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:02.583232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98e9c1-6b6f-42de-a69d-bc346e1ca161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:02.607312 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2e98e9c1-6b6f-42de-a69d-bc346e1ca161 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:02.609185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2e98e9c1-6b6f-42de-a69d-bc346e1ca161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:02.613205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab898a4-a7c7-4a75-a559-2f1f16f8ea9e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:02.629296 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:02.657597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10147911-5a08-49e8-9f4b-ccc765c49869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:02.658211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10147911-5a08-49e8-9f4b-ccc765c49869 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:02.658489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10147911-5a08-49e8-9f4b-ccc765c49869 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:02.658756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10147911-5a08-49e8-9f4b-ccc765c49869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:02.696083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:02.703755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 88/166] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:02.710053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98e9c1-6b6f-42de-a69d-bc346e1ca161 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:02.764616 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10147911-5a08-49e8-9f4b-ccc765c49869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:02.766074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10147911-5a08-49e8-9f4b-ccc765c49869 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:02.849186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98e9c1-6b6f-42de-a69d-bc346e1ca161 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:02.885825 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb1e41-2aa6-462d-87a9-2057f1eef50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:02.886668 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb1e41-2aa6-462d-87a9-2057f1eef50a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:02.886980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb1e41-2aa6-462d-87a9-2057f1eef50a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:02.887317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb1e41-2aa6-462d-87a9-2057f1eef50a None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:02.887989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb1e41-2aa6-462d-87a9-2057f1eef50a None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:02.888377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb1e41-2aa6-462d-87a9-2057f1eef50a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:02.889770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb1e41-2aa6-462d-87a9-2057f1eef50a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:02.898927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 89/167] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:02.899852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb1e41-2aa6-462d-87a9-2057f1eef50a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:02.903273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 79/168] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:02.911120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10147911-5a08-49e8-9f4b-ccc765c49869 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:02.934947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:03.044477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:03.102548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10147911-5a08-49e8-9f4b-ccc765c49869 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:03.125020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb1e41-2aa6-462d-87a9-2057f1eef50a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:03.149931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da1a37-c269-48b5-8be6-2cb2869ea83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:03.150569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da1a37-c269-48b5-8be6-2cb2869ea83e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:03.150850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da1a37-c269-48b5-8be6-2cb2869ea83e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:03.151125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da1a37-c269-48b5-8be6-2cb2869ea83e None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:03.152889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da1a37-c269-48b5-8be6-2cb2869ea83e None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:03.154362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da1a37-c269-48b5-8be6-2cb2869ea83e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:03.156055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 80/169] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:02 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:03.161562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da1a37-c269-48b5-8be6-2cb2869ea83e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:03.181886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f949667-e5df-4716-9bfa-0f84e444373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:03.182521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f949667-e5df-4716-9bfa-0f84e444373f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:03.182797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f949667-e5df-4716-9bfa-0f84e444373f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:03.183666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f949667-e5df-4716-9bfa-0f84e444373f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:03.195773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f949667-e5df-4716-9bfa-0f84e444373f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-7f949667-e5df-4716-9bfa-0f84e444373f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:03.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:03.202803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:03.203667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:03.207991 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 90/170] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:19:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:03.209013 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f949667-e5df-4716-9bfa-0f84e444373f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:03.304451 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:03.329994 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f949667-e5df-4716-9bfa-0f84e444373f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:03.344837 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da1a37-c269-48b5-8be6-2cb2869ea83e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:03.353562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:03.370064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43931035-f235-4428-9da8-4bed7678acb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:03.371224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43931035-f235-4428-9da8-4bed7678acb8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:03.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43931035-f235-4428-9da8-4bed7678acb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:03.371927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43931035-f235-4428-9da8-4bed7678acb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:03.373006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43931035-f235-4428-9da8-4bed7678acb8 None admin] RBAC: Adding query filter params (role.id=b69a8899fce14a0d839745bb76b0f441, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user.id=3d8ca4abbdac4f939233a06b4533d187) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:03.373388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43931035-f235-4428-9da8-4bed7678acb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:03.375087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43931035-f235-4428-9da8-4bed7678acb8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:03.382853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 91/171] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:03 2022] GET /identity/v3/role_assignments?user.id=3d8ca4abbdac4f939233a06b4533d187&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=b69a8899fce14a0d839745bb76b0f441 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:03.383832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43931035-f235-4428-9da8-4bed7678acb8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:03.415832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a832135-ae5b-43d9-aa2d-b244ac7b5742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:03.416503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832135-ae5b-43d9-aa2d-b244ac7b5742 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:03.416781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832135-ae5b-43d9-aa2d-b244ac7b5742 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:03.417086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832135-ae5b-43d9-aa2d-b244ac7b5742 None admin] PATH_INFO: `/v3/users/3d8ca4abbdac4f939233a06b4533d187` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:03.424723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a832135-ae5b-43d9-aa2d-b244ac7b5742 None admin] RBAC: Authorizing `identity:get_user(user_id=3d8ca4abbdac4f939233a06b4533d187)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:03.426272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a832135-ae5b-43d9-aa2d-b244ac7b5742 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:03.427972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 81/172] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:03 2022] GET /identity/v3/users/3d8ca4abbdac4f939233a06b4533d187 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:03.434207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a832135-ae5b-43d9-aa2d-b244ac7b5742 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:03.507259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43931035-f235-4428-9da8-4bed7678acb8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:03.549559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b6afe-9469-4fdc-9f4d-66c6f5ed3831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:03.550349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b6afe-9469-4fdc-9f4d-66c6f5ed3831 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:03.550636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b6afe-9469-4fdc-9f4d-66c6f5ed3831 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:03.550911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b6afe-9469-4fdc-9f4d-66c6f5ed3831 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:03.551600 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b6afe-9469-4fdc-9f4d-66c6f5ed3831 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:03.551905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b6afe-9469-4fdc-9f4d-66c6f5ed3831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:03.553775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b6afe-9469-4fdc-9f4d-66c6f5ed3831 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:03.564604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 92/173] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:03.570875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b6afe-9469-4fdc-9f4d-66c6f5ed3831 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:03.629227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:03.704039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a832135-ae5b-43d9-aa2d-b244ac7b5742 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:03.704924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b6afe-9469-4fdc-9f4d-66c6f5ed3831 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:03.762179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ef25e-d8ae-427d-a25b-301a56111144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:03.763152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ef25e-d8ae-427d-a25b-301a56111144 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:03.763642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ef25e-d8ae-427d-a25b-301a56111144 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:03.764128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ef25e-d8ae-427d-a25b-301a56111144 None admin] PATH_INFO: `/v3/users/620b714c359944a6b1d8649f781a6815` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:03.768903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ef25e-d8ae-427d-a25b-301a56111144 None admin] RBAC: Authorizing `identity:get_user(user_id=620b714c359944a6b1d8649f781a6815)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:03.770715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ef25e-d8ae-427d-a25b-301a56111144 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:03.773081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 93/174] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:03 2022] GET /identity/v3/users/620b714c359944a6b1d8649f781a6815 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:03.779090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ef25e-d8ae-427d-a25b-301a56111144 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:03.800148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe90715-637c-4f04-bbfb-b26c4d680e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:03.800768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe90715-637c-4f04-bbfb-b26c4d680e5b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:03.801045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe90715-637c-4f04-bbfb-b26c4d680e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:03.801314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe90715-637c-4f04-bbfb-b26c4d680e5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:03.927544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:03.927544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe90715-637c-4f04-bbfb-b26c4d680e5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:03.927544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe90715-637c-4f04-bbfb-b26c4d680e5b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:03.939488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ef25e-d8ae-427d-a25b-301a56111144 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:03.964915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:04.020726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86a95e-b76c-4594-afc3-820163996035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:04.021382 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86a95e-b76c-4594-afc3-820163996035 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:04.021668 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86a95e-b76c-4594-afc3-820163996035 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:04.021946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86a95e-b76c-4594-afc3-820163996035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:04.072519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 82/175] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 22:19:04.080991 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe90715-637c-4f04-bbfb-b26c4d680e5b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:04.160991 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86a95e-b76c-4594-afc3-820163996035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:04.162488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86a95e-b76c-4594-afc3-820163996035 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:04.180443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:04.301040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 94/176] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 22:19:04.303945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe90715-637c-4f04-bbfb-b26c4d680e5b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:04.306719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86a95e-b76c-4594-afc3-820163996035 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:04.321429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:04.410242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86a95e-b76c-4594-afc3-820163996035 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:04.413298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0664119-cfec-4a05-abf4-9a017dfd6b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:04.413941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0664119-cfec-4a05-abf4-9a017dfd6b58 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:04.414222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0664119-cfec-4a05-abf4-9a017dfd6b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:04.414493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0664119-cfec-4a05-abf4-9a017dfd6b58 None admin] PATH_INFO: `/v3/projects/26b9cf71b4654229b9cea727c2e020a1` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:04.420730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0664119-cfec-4a05-abf4-9a017dfd6b58 None admin] RBAC: Authorizing `identity:get_project(project_id=26b9cf71b4654229b9cea727c2e020a1)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:04.422208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0664119-cfec-4a05-abf4-9a017dfd6b58 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:04.423881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 83/177] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:04 2022] GET /identity/v3/projects/26b9cf71b4654229b9cea727c2e020a1 => generated 312 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:04.429916 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0664119-cfec-4a05-abf4-9a017dfd6b58 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:04.446316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf533209-0572-4622-b57d-f4f718fc0290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:04.446875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf533209-0572-4622-b57d-f4f718fc0290 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:04.447155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf533209-0572-4622-b57d-f4f718fc0290 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:04.447429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf533209-0572-4622-b57d-f4f718fc0290 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:04.448769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf533209-0572-4622-b57d-f4f718fc0290 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:04.450225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf533209-0572-4622-b57d-f4f718fc0290 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:04.452058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 95/178] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:04 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:04.457707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf533209-0572-4622-b57d-f4f718fc0290 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:04.544588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:04.560922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf533209-0572-4622-b57d-f4f718fc0290 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:04.596860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0664119-cfec-4a05-abf4-9a017dfd6b58 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:04.608739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455fd70-b57b-4cab-87d6-56a494452d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:04.609371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455fd70-b57b-4cab-87d6-56a494452d8f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:04.609653 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455fd70-b57b-4cab-87d6-56a494452d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:04.609926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455fd70-b57b-4cab-87d6-56a494452d8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:04.610669 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455fd70-b57b-4cab-87d6-56a494452d8f None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user.id=620b714c359944a6b1d8649f781a6815) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:04.610987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455fd70-b57b-4cab-87d6-56a494452d8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:04.611785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:04.612482 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455fd70-b57b-4cab-87d6-56a494452d8f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:04.618883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 96/179] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:04 2022] GET /identity/v3/role_assignments?user.id=620b714c359944a6b1d8649f781a6815&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=108664305d2546949f7327b65d4cb667 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:04.646079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db69f9b-c933-41bc-9ead-4f001edef4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:04.646821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db69f9b-c933-41bc-9ead-4f001edef4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:04.647101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db69f9b-c933-41bc-9ead-4f001edef4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:04.647428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db69f9b-c933-41bc-9ead-4f001edef4d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:04.648216 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db69f9b-c933-41bc-9ead-4f001edef4d6 None admin] RBAC: Adding query filter params (role.id=87ab4761555e43419d0f23a0f31114f3, scope.project.id=26b9cf71b4654229b9cea727c2e020a1, user.id=3d8ca4abbdac4f939233a06b4533d187) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:04.648576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db69f9b-c933-41bc-9ead-4f001edef4d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:04.650090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db69f9b-c933-41bc-9ead-4f001edef4d6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:04.657934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 84/180] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:04 2022] GET /identity/v3/role_assignments?user.id=3d8ca4abbdac4f939233a06b4533d187&scope.project.id=26b9cf71b4654229b9cea727c2e020a1&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:05.131818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185ecce-e0f1-4f2c-86eb-c78b5b1dca3d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.132231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185ecce-e0f1-4f2c-86eb-c78b5b1dca3d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.132407 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185ecce-e0f1-4f2c-86eb-c78b5b1dca3d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.133743 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 97/181] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:05.139833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1f259-5799-49f3-ab5a-89d7191efafc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.140162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1f259-5799-49f3-ab5a-89d7191efafc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.140441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1f259-5799-49f3-ab5a-89d7191efafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.161900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e4a1f259-5799-49f3-ab5a-89d7191efafc None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:05.163103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e4a1f259-5799-49f3-ab5a-89d7191efafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:05.191617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b70c-836a-4759-b1ab-33fe7847099f None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.191892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b70c-836a-4759-b1ab-33fe7847099f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.192191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b70c-836a-4759-b1ab-33fe7847099f None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.193659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 98/182] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:05.199472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2280246-216c-468e-a875-749a66a88d36 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.199767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2280246-216c-468e-a875-749a66a88d36 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.200175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2280246-216c-468e-a875-749a66a88d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.228189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c2280246-216c-468e-a875-749a66a88d36 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:05.229789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c2280246-216c-468e-a875-749a66a88d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:05.267443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 85/183] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:05.285256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1f259-5799-49f3-ab5a-89d7191efafc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:05.337494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 99/184] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:05.364477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2280246-216c-468e-a875-749a66a88d36 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:05.405042 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1f259-5799-49f3-ab5a-89d7191efafc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:05.417339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:05.449759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab9308-0158-4eeb-bf35-cd7482bbb11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:05.450627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab9308-0158-4eeb-bf35-cd7482bbb11d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.450735 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab9308-0158-4eeb-bf35-cd7482bbb11d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.450922 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab9308-0158-4eeb-bf35-cd7482bbb11d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.458967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:05.460997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab9308-0158-4eeb-bf35-cd7482bbb11d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-8bab9308-0158-4eeb-bf35-cd7482bbb11d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:05.466856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:05.468060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:05.469049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:05.472172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 86/185] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:05.479587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9ce5-d357-42a8-99d3-bf0899f0d9c4 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.479587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9ce5-d357-42a8-99d3-bf0899f0d9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.479587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9ce5-d357-42a8-99d3-bf0899f0d9c4 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.479587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 87/186] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:05.479587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d9ce5-d357-42a8-99d3-bf0899f0d9c4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:05.487718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2280246-216c-468e-a875-749a66a88d36 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:05.525209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0614202-ae2f-4c10-a238-ab240204ac99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:05.525844 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0614202-ae2f-4c10-a238-ab240204ac99 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.526417 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0614202-ae2f-4c10-a238-ab240204ac99 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.527329 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0614202-ae2f-4c10-a238-ab240204ac99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.538315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0614202-ae2f-4c10-a238-ab240204ac99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f0614202-ae2f-4c10-a238-ab240204ac99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:05.543245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:05.544172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:05.544957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 100/187] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:05.545724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba46dc-6e4d-4896-80c6-be769f485fcc None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.545935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba46dc-6e4d-4896-80c6-be769f485fcc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.546182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba46dc-6e4d-4896-80c6-be769f485fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.564087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6bba46dc-6e4d-4896-80c6-be769f485fcc None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:05.565284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6bba46dc-6e4d-4896-80c6-be769f485fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:05.587064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d9ce5-d357-42a8-99d3-bf0899f0d9c4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:05.621144 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0025f-c691-4a98-8ba7-f8347695cab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:05.621742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0025f-c691-4a98-8ba7-f8347695cab5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.622023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0025f-c691-4a98-8ba7-f8347695cab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.622299 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0025f-c691-4a98-8ba7-f8347695cab5 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.622958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0025f-c691-4a98-8ba7-f8347695cab5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:05.623261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0025f-c691-4a98-8ba7-f8347695cab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:05.624669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0025f-c691-4a98-8ba7-f8347695cab5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:05.629929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 101/188] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:05.631161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2f39a-8729-4bca-a29a-1bcf8794840b None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.631447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2f39a-8729-4bca-a29a-1bcf8794840b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.631725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2f39a-8729-4bca-a29a-1bcf8794840b None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.636436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 102/189] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:05.636436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2f39a-8729-4bca-a29a-1bcf8794840b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:05.640053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 88/190] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:05.640306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e3f4a-5cda-4eb5-bf71-4ddb68fcd2f6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.640593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e3f4a-5cda-4eb5-bf71-4ddb68fcd2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.640869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e3f4a-5cda-4eb5-bf71-4ddb68fcd2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.661617 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a08e3f4a-5cda-4eb5-bf71-4ddb68fcd2f6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:05.662800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a08e3f4a-5cda-4eb5-bf71-4ddb68fcd2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:05.686172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:05.710556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:05.728481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 89/191] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:05.729492 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e3f4a-5cda-4eb5-bf71-4ddb68fcd2f6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:05.765250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2f39a-8729-4bca-a29a-1bcf8794840b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:05.798973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f239cd-e84b-4819-821d-36d3b7190654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:05.799575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f239cd-e84b-4819-821d-36d3b7190654 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.799852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f239cd-e84b-4819-821d-36d3b7190654 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.800154 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f239cd-e84b-4819-821d-36d3b7190654 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.800819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f239cd-e84b-4819-821d-36d3b7190654 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:05.801121 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f239cd-e84b-4819-821d-36d3b7190654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:05.802514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f239cd-e84b-4819-821d-36d3b7190654 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:05.811255 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 103/192] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:05.812062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f239cd-e84b-4819-821d-36d3b7190654 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:05.827418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e3f4a-5cda-4eb5-bf71-4ddb68fcd2f6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:05.860580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61bff0-cf5e-4847-a566-f5aaa4c3cbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:05.861203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61bff0-cf5e-4847-a566-f5aaa4c3cbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.861477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61bff0-cf5e-4847-a566-f5aaa4c3cbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.861745 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61bff0-cf5e-4847-a566-f5aaa4c3cbb3 None admin] PATH_INFO: `/v3/users/fcb20c7bffdd41e8a68646bd5c99d3d4` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.863231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61bff0-cf5e-4847-a566-f5aaa4c3cbb3 None admin] RBAC: Authorizing `identity:get_user(user_id=fcb20c7bffdd41e8a68646bd5c99d3d4)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:05.864699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61bff0-cf5e-4847-a566-f5aaa4c3cbb3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:05.866251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 90/193] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:05 2022] GET /identity/v3/users/fcb20c7bffdd41e8a68646bd5c99d3d4 => generated 289 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:05.866998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61bff0-cf5e-4847-a566-f5aaa4c3cbb3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:05.894939 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:05.913735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f239cd-e84b-4819-821d-36d3b7190654 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:05.939822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:05.947260 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d8839-be2c-4bfd-9fab-6424d15e8cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:05.947937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d8839-be2c-4bfd-9fab-6424d15e8cbc None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:05.948245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d8839-be2c-4bfd-9fab-6424d15e8cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:05.952906 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d8839-be2c-4bfd-9fab-6424d15e8cbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:05.959809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d8839-be2c-4bfd-9fab-6424d15e8cbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-447d8839-be2c-4bfd-9fab-6424d15e8cbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:05.965083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:05.966028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:05.966842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 104/194] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:19:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:05.967499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d8839-be2c-4bfd-9fab-6424d15e8cbc None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:05.968160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61bff0-cf5e-4847-a566-f5aaa4c3cbb3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:06.003150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636ae56-c70e-4db7-ae33-2158ca0b1529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:06.003774 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636ae56-c70e-4db7-ae33-2158ca0b1529 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:06.004085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636ae56-c70e-4db7-ae33-2158ca0b1529 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:06.004363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636ae56-c70e-4db7-ae33-2158ca0b1529 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:06.005465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636ae56-c70e-4db7-ae33-2158ca0b1529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:06.006911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636ae56-c70e-4db7-ae33-2158ca0b1529 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:06.008506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 91/195] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:06.009344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636ae56-c70e-4db7-ae33-2158ca0b1529 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:06.075459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d8839-be2c-4bfd-9fab-6424d15e8cbc None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:06.106578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:06.112457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636ae56-c70e-4db7-ae33-2158ca0b1529 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:06.114115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4fa7a-e5ee-4df5-a8da-fc6453b7a06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:06.114749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4fa7a-e5ee-4df5-a8da-fc6453b7a06d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:06.115028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4fa7a-e5ee-4df5-a8da-fc6453b7a06d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:06.118696 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4fa7a-e5ee-4df5-a8da-fc6453b7a06d None admin] PATH_INFO: `/v3/users/bc2dc60490bc466c845c20f678cf0d87` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:06.129496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4fa7a-e5ee-4df5-a8da-fc6453b7a06d None admin] RBAC: Authorizing `identity:get_user(user_id=bc2dc60490bc466c845c20f678cf0d87)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:06.131151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4fa7a-e5ee-4df5-a8da-fc6453b7a06d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:06.132796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 105/196] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:05 2022] GET /identity/v3/users/bc2dc60490bc466c845c20f678cf0d87 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:06.137628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4fa7a-e5ee-4df5-a8da-fc6453b7a06d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:06.146638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772d424-8cd9-457b-81fa-022f65cfffa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:06.147244 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772d424-8cd9-457b-81fa-022f65cfffa6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:06.147518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772d424-8cd9-457b-81fa-022f65cfffa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:06.147788 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772d424-8cd9-457b-81fa-022f65cfffa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:06.179545 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:06.213443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772d424-8cd9-457b-81fa-022f65cfffa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:06.215501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772d424-8cd9-457b-81fa-022f65cfffa6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:06.233719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4fa7a-e5ee-4df5-a8da-fc6453b7a06d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:06.266628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e5663-3bfe-4696-997a-9f8daee3510d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:06.267223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e5663-3bfe-4696-997a-9f8daee3510d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:06.267503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e5663-3bfe-4696-997a-9f8daee3510d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:06.267776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e5663-3bfe-4696-997a-9f8daee3510d None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:06.268465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e5663-3bfe-4696-997a-9f8daee3510d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:06.268772 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e5663-3bfe-4696-997a-9f8daee3510d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:06.270131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e5663-3bfe-4696-997a-9f8daee3510d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:06.277956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 106/197] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:06.280817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e5663-3bfe-4696-997a-9f8daee3510d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:06.307206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:06.313661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 92/198] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:06.314460 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772d424-8cd9-457b-81fa-022f65cfffa6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:06.381633 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e5663-3bfe-4696-997a-9f8daee3510d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:06.390852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:06.412033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772d424-8cd9-457b-81fa-022f65cfffa6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:06.415473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d24fe-7985-4c68-bd32-043a6090e230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:06.416134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d24fe-7985-4c68-bd32-043a6090e230 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:06.416682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d24fe-7985-4c68-bd32-043a6090e230 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:06.416968 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d24fe-7985-4c68-bd32-043a6090e230 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:06.417888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d24fe-7985-4c68-bd32-043a6090e230 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:06.419642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d24fe-7985-4c68-bd32-043a6090e230 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:06.444560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fcba5-d44a-49db-89ad-c0de9f8fca90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:06.452891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fcba5-d44a-49db-89ad-c0de9f8fca90 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:06.452891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fcba5-d44a-49db-89ad-c0de9f8fca90 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:06.452891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fcba5-d44a-49db-89ad-c0de9f8fca90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:06.502802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 107/199] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:19:06 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:19:06.503687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d24fe-7985-4c68-bd32-043a6090e230 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:06.513581 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4fcba5-d44a-49db-89ad-c0de9f8fca90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:06.515107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4fcba5-d44a-49db-89ad-c0de9f8fca90 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:06.524519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:06.622435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d24fe-7985-4c68-bd32-043a6090e230 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:06.643864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 93/200] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:06.649764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fcba5-d44a-49db-89ad-c0de9f8fca90 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:06.662010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c036c-5e16-41c5-b26b-f5414ec6664a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:06.662653 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c036c-5e16-41c5-b26b-f5414ec6664a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:06.662933 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c036c-5e16-41c5-b26b-f5414ec6664a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:06.663209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c036c-5e16-41c5-b26b-f5414ec6664a None admin] PATH_INFO: `/v3/users/bc2dc60490bc466c845c20f678cf0d87` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:06.664682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c036c-5e16-41c5-b26b-f5414ec6664a None admin] RBAC: Authorizing `identity:get_user(user_id=bc2dc60490bc466c845c20f678cf0d87)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:06.666130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c036c-5e16-41c5-b26b-f5414ec6664a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:06.668491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 108/201] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:06 2022] GET /identity/v3/users/bc2dc60490bc466c845c20f678cf0d87 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:06.669187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c036c-5e16-41c5-b26b-f5414ec6664a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:06.705050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:06.772178 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:06.779420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fcba5-d44a-49db-89ad-c0de9f8fca90 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:06.789977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c036c-5e16-41c5-b26b-f5414ec6664a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:06.826074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928eaaba-4563-48ee-9aa7-51f25b6b3b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:06.826711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eaaba-4563-48ee-9aa7-51f25b6b3b6c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:06.826994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eaaba-4563-48ee-9aa7-51f25b6b3b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:06.827272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eaaba-4563-48ee-9aa7-51f25b6b3b6c None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:06.828472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928eaaba-4563-48ee-9aa7-51f25b6b3b6c None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:06.829925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928eaaba-4563-48ee-9aa7-51f25b6b3b6c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:06.831574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 94/202] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:06 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:06.836048 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928eaaba-4563-48ee-9aa7-51f25b6b3b6c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:06.840379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c33b33-8a89-431a-a2d9-1df2ce582ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:06.841033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c33b33-8a89-431a-a2d9-1df2ce582ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:06.841314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c33b33-8a89-431a-a2d9-1df2ce582ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:06.841591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c33b33-8a89-431a-a2d9-1df2ce582ec8 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:06.843365 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c33b33-8a89-431a-a2d9-1df2ce582ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:06.845354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c33b33-8a89-431a-a2d9-1df2ce582ec8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:06.847325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 109/203] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:06 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:06.849155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c33b33-8a89-431a-a2d9-1df2ce582ec8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:06.954665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928eaaba-4563-48ee-9aa7-51f25b6b3b6c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:06.978856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:06.982733 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c33b33-8a89-431a-a2d9-1df2ce582ec8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:06.998635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e0d09-b8ab-45e0-bd17-1704d67ed654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:06.999553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e0d09-b8ab-45e0-bd17-1704d67ed654 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:06.999838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e0d09-b8ab-45e0-bd17-1704d67ed654 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:07.000158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e0d09-b8ab-45e0-bd17-1704d67ed654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:07.021226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcc88c-777e-43a5-bef9-bb02238f0b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:07.021987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcc88c-777e-43a5-bef9-bb02238f0b2a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:07.022396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcc88c-777e-43a5-bef9-bb02238f0b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:07.022681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcc88c-777e-43a5-bef9-bb02238f0b2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:07.023492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcc88c-777e-43a5-bef9-bb02238f0b2a None admin] RBAC: Adding query filter params (role.id=0f9e8507916f478c80987f7c7fd3c3e9, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7, user.id=fcb20c7bffdd41e8a68646bd5c99d3d4) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:07.023871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcc88c-777e-43a5-bef9-bb02238f0b2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:07.025533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcc88c-777e-43a5-bef9-bb02238f0b2a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:07.036169 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 110/204] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:06 2022] GET /identity/v3/role_assignments?user.id=fcb20c7bffdd41e8a68646bd5c99d3d4&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=0f9e8507916f478c80987f7c7fd3c3e9 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:07.037224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcc88c-777e-43a5-bef9-bb02238f0b2a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:07.068464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:07.084590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e0d09-b8ab-45e0-bd17-1704d67ed654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:07.086652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e0d09-b8ab-45e0-bd17-1704d67ed654 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:07.230187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 95/205] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:07.235954 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e0d09-b8ab-45e0-bd17-1704d67ed654 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:07.238339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcc88c-777e-43a5-bef9-bb02238f0b2a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:07.282622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac728eb-f275-4f7e-acd0-e5ce64b45ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:07.283162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac728eb-f275-4f7e-acd0-e5ce64b45ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:07.283442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac728eb-f275-4f7e-acd0-e5ce64b45ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:07.283717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac728eb-f275-4f7e-acd0-e5ce64b45ae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:07.284494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac728eb-f275-4f7e-acd0-e5ce64b45ae9 None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3, user.id=bc2dc60490bc466c845c20f678cf0d87) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:07.284808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac728eb-f275-4f7e-acd0-e5ce64b45ae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:07.286762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac728eb-f275-4f7e-acd0-e5ce64b45ae9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:07.294920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 111/206] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:07 2022] GET /identity/v3/role_assignments?user.id=bc2dc60490bc466c845c20f678cf0d87&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=108664305d2546949f7327b65d4cb667 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:07.320423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:07.358513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e0d09-b8ab-45e0-bd17-1704d67ed654 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:07.397694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0da2cf-4668-4f5f-8e0f-0e71cca15c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:07.398732 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0da2cf-4668-4f5f-8e0f-0e71cca15c06 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:07.399174 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0da2cf-4668-4f5f-8e0f-0e71cca15c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:07.399585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0da2cf-4668-4f5f-8e0f-0e71cca15c06 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:07.400939 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0da2cf-4668-4f5f-8e0f-0e71cca15c06 None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:07.403086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0da2cf-4668-4f5f-8e0f-0e71cca15c06 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:07.405058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 96/207] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:07 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:07.411143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac728eb-f275-4f7e-acd0-e5ce64b45ae9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:07.461697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:07.540442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac728eb-f275-4f7e-acd0-e5ce64b45ae9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:07.580177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e12d8a-92a1-4c2a-a533-480cf7510b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:07.580795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e12d8a-92a1-4c2a-a533-480cf7510b2a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:07.581078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e12d8a-92a1-4c2a-a533-480cf7510b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:07.581356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e12d8a-92a1-4c2a-a533-480cf7510b2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:07.582096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e12d8a-92a1-4c2a-a533-480cf7510b2a None admin] RBAC: Adding query filter params (role.id=b69a8899fce14a0d839745bb76b0f441, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3, user.id=bc2dc60490bc466c845c20f678cf0d87) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:07.582405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e12d8a-92a1-4c2a-a533-480cf7510b2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:07.583972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e12d8a-92a1-4c2a-a533-480cf7510b2a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:07.590628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 112/208] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:07 2022] GET /identity/v3/role_assignments?user.id=bc2dc60490bc466c845c20f678cf0d87&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=b69a8899fce14a0d839745bb76b0f441 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:08.785745 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d135-e0c2-4a04-a64e-edddc9db7590 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:08.786289 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d135-e0c2-4a04-a64e-edddc9db7590 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:08.786289 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d135-e0c2-4a04-a64e-edddc9db7590 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:08.787757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 97/209] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:08.793608 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c67fcd-b2c0-44c8-9a55-9cc891cab1ed None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:08.794007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c67fcd-b2c0-44c8-9a55-9cc891cab1ed None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:08.794185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c67fcd-b2c0-44c8-9a55-9cc891cab1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:08.812442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b9c67fcd-b2c0-44c8-9a55-9cc891cab1ed None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:08.813634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b9c67fcd-b2c0-44c8-9a55-9cc891cab1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:08.823203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac9c0-433f-4a13-8af9-51d71c1ac1cb None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:08.823203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac9c0-433f-4a13-8af9-51d71c1ac1cb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:08.823203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac9c0-433f-4a13-8af9-51d71c1ac1cb None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:08.823203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 98/210] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:08.832159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679e0f2-7218-4a5a-8397-fcdba053e793 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:08.832977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679e0f2-7218-4a5a-8397-fcdba053e793 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:08.833770 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679e0f2-7218-4a5a-8397-fcdba053e793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:08.857611 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0679e0f2-7218-4a5a-8397-fcdba053e793 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:08.859286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0679e0f2-7218-4a5a-8397-fcdba053e793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:08.899527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 113/211] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:08.923809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c67fcd-b2c0-44c8-9a55-9cc891cab1ed None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:08.941437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 99/212] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:08.954078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:08.968325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679e0f2-7218-4a5a-8397-fcdba053e793 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:09.036158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c67fcd-b2c0-44c8-9a55-9cc891cab1ed None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:09.036938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:09.073121 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd314ae-1887-426a-ad75-a4b30c81af85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:09.074240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd314ae-1887-426a-ad75-a4b30c81af85 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.074772 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd314ae-1887-426a-ad75-a4b30c81af85 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.075224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd314ae-1887-426a-ad75-a4b30c81af85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.080552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679e0f2-7218-4a5a-8397-fcdba053e793 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:09.086705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd314ae-1887-426a-ad75-a4b30c81af85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-cbd314ae-1887-426a-ad75-a4b30c81af85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:09.092809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:09.093701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:09.094476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:09.096698 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 114/213] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:09.102827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd314ae-1887-426a-ad75-a4b30c81af85 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:09.117119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9a212-d121-48b7-824b-1c55696a3717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:09.117922 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9a212-d121-48b7-824b-1c55696a3717 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.118273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9a212-d121-48b7-824b-1c55696a3717 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.118635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9a212-d121-48b7-824b-1c55696a3717 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.120147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9a212-d121-48b7-824b-1c55696a3717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:09.121891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9a212-d121-48b7-824b-1c55696a3717 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:09.128295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 100/214] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:09.129387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9a212-d121-48b7-824b-1c55696a3717 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:09.195964 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:09.217262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd314ae-1887-426a-ad75-a4b30c81af85 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:09.240445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9a212-d121-48b7-824b-1c55696a3717 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:09.255341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943ea8c-73c3-4858-981b-99042ce5fba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:09.256300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943ea8c-73c3-4858-981b-99042ce5fba6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.256751 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943ea8c-73c3-4858-981b-99042ce5fba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.257227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943ea8c-73c3-4858-981b-99042ce5fba6 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.258097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943ea8c-73c3-4858-981b-99042ce5fba6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:09.258683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943ea8c-73c3-4858-981b-99042ce5fba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:09.260374 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943ea8c-73c3-4858-981b-99042ce5fba6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:09.270754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 115/215] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:09.275733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:09.278262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943ea8c-73c3-4858-981b-99042ce5fba6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:09.279072 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68945d55-cb5f-46fe-a38c-f8cb5092b6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:09.279679 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68945d55-cb5f-46fe-a38c-f8cb5092b6c1 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.279953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68945d55-cb5f-46fe-a38c-f8cb5092b6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.280266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68945d55-cb5f-46fe-a38c-f8cb5092b6c1 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.280912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68945d55-cb5f-46fe-a38c-f8cb5092b6c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:09.282574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68945d55-cb5f-46fe-a38c-f8cb5092b6c1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:09.382879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943ea8c-73c3-4858-981b-99042ce5fba6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:09.383766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 101/216] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:19:09 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:19:09.389505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c4719-3e39-4689-bf34-57bb4bc75c19 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.392808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c4719-3e39-4689-bf34-57bb4bc75c19 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.392808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c4719-3e39-4689-bf34-57bb4bc75c19 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.392808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 102/217] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:09.397249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf2660-f472-4866-a8a1-de4d64c5b9cf None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.397532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf2660-f472-4866-a8a1-de4d64c5b9cf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.397809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf2660-f472-4866-a8a1-de4d64c5b9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.420163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-eeaf2660-f472-4866-a8a1-de4d64c5b9cf None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:09.421402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-eeaf2660-f472-4866-a8a1-de4d64c5b9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:09.444218 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fec277-88f5-4f90-9864-33170e5be5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:09.445978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fec277-88f5-4f90-9864-33170e5be5dc None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.446794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fec277-88f5-4f90-9864-33170e5be5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.447305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fec277-88f5-4f90-9864-33170e5be5dc None admin] PATH_INFO: `/v3/users/ca1046f934824e73af9e1f0e9415b4c9` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.469876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fec277-88f5-4f90-9864-33170e5be5dc None admin] RBAC: Authorizing `identity:get_user(user_id=ca1046f934824e73af9e1f0e9415b4c9)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:09.472445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fec277-88f5-4f90-9864-33170e5be5dc None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:09.475210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 116/218] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:09 2022] GET /identity/v3/users/ca1046f934824e73af9e1f0e9415b4c9 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:09.481206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fec277-88f5-4f90-9864-33170e5be5dc None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:09.484851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:09.504816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 103/219] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:09.528433 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf2660-f472-4866-a8a1-de4d64c5b9cf None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:09.630963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fec277-88f5-4f90-9864-33170e5be5dc None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:09.672304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01d26c-5c96-41d9-9998-0e51ad93ca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:09.673405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01d26c-5c96-41d9-9998-0e51ad93ca8e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.674855 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01d26c-5c96-41d9-9998-0e51ad93ca8e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.675604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01d26c-5c96-41d9-9998-0e51ad93ca8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.679931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf2660-f472-4866-a8a1-de4d64c5b9cf None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:09.701209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:09.727982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63cb66-9f9e-4573-986f-63d8398ecdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:09.728666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63cb66-9f9e-4573-986f-63d8398ecdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.728981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63cb66-9f9e-4573-986f-63d8398ecdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.729235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63cb66-9f9e-4573-986f-63d8398ecdd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.742910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63cb66-9f9e-4573-986f-63d8398ecdd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-eb63cb66-9f9e-4573-986f-63d8398ecdd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:09.749857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:09.751155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:09.751976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 104/220] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:19:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:09.753030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f02b5-7947-4620-ad0c-e012eb07a50f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.753345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f02b5-7947-4620-ad0c-e012eb07a50f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.753669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f02b5-7947-4620-ad0c-e012eb07a50f None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.760387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 105/221] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:09.760528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f02b5-7947-4620-ad0c-e012eb07a50f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:09.766648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01d26c-5c96-41d9-9998-0e51ad93ca8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:09.767977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01d26c-5c96-41d9-9998-0e51ad93ca8e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:09.770080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:09.881045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 117/222] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 22:19:09.887623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cfcb0-2507-4a90-8d34-eaf2c114ea99 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.887924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cfcb0-2507-4a90-8d34-eaf2c114ea99 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.888233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cfcb0-2507-4a90-8d34-eaf2c114ea99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.890728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f02b5-7947-4620-ad0c-e012eb07a50f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:09.908738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6a2cfcb0-2507-4a90-8d34-eaf2c114ea99 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:09.910069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6a2cfcb0-2507-4a90-8d34-eaf2c114ea99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:09.931242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8f067-6396-4e7e-a3b6-28deef8ae138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:09.931242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f067-6396-4e7e-a3b6-28deef8ae138 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:09.931242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f067-6396-4e7e-a3b6-28deef8ae138 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:09.931242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f067-6396-4e7e-a3b6-28deef8ae138 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:09.931626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8f067-6396-4e7e-a3b6-28deef8ae138 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:09.931626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8f067-6396-4e7e-a3b6-28deef8ae138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:09.933124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8f067-6396-4e7e-a3b6-28deef8ae138 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:09.947734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 106/223] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:09.949511 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8f067-6396-4e7e-a3b6-28deef8ae138 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:09.989817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 118/224] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:09.990686 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cfcb0-2507-4a90-8d34-eaf2c114ea99 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:09.998720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:10.048061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:10.082155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8f067-6396-4e7e-a3b6-28deef8ae138 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:10.122045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ac021-a1e3-4441-ba3e-f1f721ce903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:10.122828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ac021-a1e3-4441-ba3e-f1f721ce903a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:10.123387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ac021-a1e3-4441-ba3e-f1f721ce903a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:10.123537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ac021-a1e3-4441-ba3e-f1f721ce903a None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:10.124801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ac021-a1e3-4441-ba3e-f1f721ce903a None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:10.126440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ac021-a1e3-4441-ba3e-f1f721ce903a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:10.128482 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 107/225] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:09 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:10.132068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ac021-a1e3-4441-ba3e-f1f721ce903a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:10.134096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cfcb0-2507-4a90-8d34-eaf2c114ea99 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:10.176002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b2a66-b53c-47e4-8400-e0e7f895e2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:10.176670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b2a66-b53c-47e4-8400-e0e7f895e2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:10.176952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b2a66-b53c-47e4-8400-e0e7f895e2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:10.177227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b2a66-b53c-47e4-8400-e0e7f895e2b6 None admin] PATH_INFO: `/v3/users/bc2dc60490bc466c845c20f678cf0d87` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:10.178377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b2a66-b53c-47e4-8400-e0e7f895e2b6 None admin] RBAC: Authorizing `identity:get_user(user_id=bc2dc60490bc466c845c20f678cf0d87)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:10.180394 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b2a66-b53c-47e4-8400-e0e7f895e2b6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:10.182765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 119/226] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:09 2022] GET /identity/v3/users/bc2dc60490bc466c845c20f678cf0d87 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:10.183287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b2a66-b53c-47e4-8400-e0e7f895e2b6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:10.249339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ac021-a1e3-4441-ba3e-f1f721ce903a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:10.257861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:10.288492 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a6f87-5463-4742-ab13-59be569e2972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:10.289582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6f87-5463-4742-ab13-59be569e2972 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:10.290148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6f87-5463-4742-ab13-59be569e2972 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:10.290677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6f87-5463-4742-ab13-59be569e2972 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:10.297638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:10.301260 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b2a66-b53c-47e4-8400-e0e7f895e2b6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:10.304322 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a6f87-5463-4742-ab13-59be569e2972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d24a6f87-5463-4742-ab13-59be569e2972 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:10.310231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:10.311118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:10.311883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:10.314297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 108/227] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:19:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:10.315459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a6f87-5463-4742-ab13-59be569e2972 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:10.340916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2ea86-4dba-42cf-958c-97952438d278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:10.341536 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2ea86-4dba-42cf-958c-97952438d278 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:10.341815 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2ea86-4dba-42cf-958c-97952438d278 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:10.342092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2ea86-4dba-42cf-958c-97952438d278 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:10.342898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2ea86-4dba-42cf-958c-97952438d278 None admin] RBAC: Adding query filter params (role.id=87ab4761555e43419d0f23a0f31114f3, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3, user.id=ca1046f934824e73af9e1f0e9415b4c9) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:10.343196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2ea86-4dba-42cf-958c-97952438d278 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:10.344778 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2ea86-4dba-42cf-958c-97952438d278 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:10.356578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 120/228] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:10 2022] GET /identity/v3/role_assignments?user.id=ca1046f934824e73af9e1f0e9415b4c9&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:10.358041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2ea86-4dba-42cf-958c-97952438d278 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:10.430766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a6f87-5463-4742-ab13-59be569e2972 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:10.471505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f49af-9a7d-49e3-a303-4fb38ad2fa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:10.472691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f49af-9a7d-49e3-a303-4fb38ad2fa8c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:10.473211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f49af-9a7d-49e3-a303-4fb38ad2fa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:10.473694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f49af-9a7d-49e3-a303-4fb38ad2fa8c None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:10.475203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f49af-9a7d-49e3-a303-4fb38ad2fa8c None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:10.477082 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f49af-9a7d-49e3-a303-4fb38ad2fa8c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:10.479040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 109/229] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:10 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:10.485101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f49af-9a7d-49e3-a303-4fb38ad2fa8c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:10.530396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:10.589473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2ea86-4dba-42cf-958c-97952438d278 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:10.614634 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f49af-9a7d-49e3-a303-4fb38ad2fa8c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:10.639670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f352a0e-9e06-4085-a744-fc62a0cc692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:10.640335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f352a0e-9e06-4085-a744-fc62a0cc692c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:10.640624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f352a0e-9e06-4085-a744-fc62a0cc692c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:10.640905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f352a0e-9e06-4085-a744-fc62a0cc692c None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:10.641594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f352a0e-9e06-4085-a744-fc62a0cc692c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:10.641896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f352a0e-9e06-4085-a744-fc62a0cc692c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:10.643405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f352a0e-9e06-4085-a744-fc62a0cc692c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:10.656392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8e3e3-588c-42d3-ba1b-54690a9891a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:10.657660 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8e3e3-588c-42d3-ba1b-54690a9891a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:10.658140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8e3e3-588c-42d3-ba1b-54690a9891a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:10.658608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8e3e3-588c-42d3-ba1b-54690a9891a3 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/users/bc2dc60490bc466c845c20f678cf0d87/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:10.660777 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8e3e3-588c-42d3-ba1b-54690a9891a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3, user_id=bc2dc60490bc466c845c20f678cf0d87, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:10.662226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 121/230] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:10.662719 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8e3e3-588c-42d3-ba1b-54690a9891a3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:10.667873 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f352a0e-9e06-4085-a744-fc62a0cc692c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:10.701347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:10.738603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 110/231] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:19:10 2022] PUT /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/users/bc2dc60490bc466c845c20f678cf0d87/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:19:10.789895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f352a0e-9e06-4085-a744-fc62a0cc692c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:10.829947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54844b0c-e452-49d0-a354-561d99a0548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:10.830607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54844b0c-e452-49d0-a354-561d99a0548d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:10.830890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54844b0c-e452-49d0-a354-561d99a0548d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:10.831171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54844b0c-e452-49d0-a354-561d99a0548d None admin] PATH_INFO: `/v3/users/bc2dc60490bc466c845c20f678cf0d87` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:10.832403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54844b0c-e452-49d0-a354-561d99a0548d None admin] RBAC: Authorizing `identity:get_user(user_id=bc2dc60490bc466c845c20f678cf0d87)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:10.834031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54844b0c-e452-49d0-a354-561d99a0548d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:10.836055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 122/232] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:10 2022] GET /identity/v3/users/bc2dc60490bc466c845c20f678cf0d87 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:10.841509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8e3e3-588c-42d3-ba1b-54690a9891a3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:10.994070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:11.002584 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8e3e3-588c-42d3-ba1b-54690a9891a3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:11.039993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbf2ea-f07b-4441-8e0e-1be95da48aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:11.041049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf2ea-f07b-4441-8e0e-1be95da48aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:11.041520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf2ea-f07b-4441-8e0e-1be95da48aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:11.042498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf2ea-f07b-4441-8e0e-1be95da48aa9 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:11.043814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbf2ea-f07b-4441-8e0e-1be95da48aa9 None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:11.045439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbf2ea-f07b-4441-8e0e-1be95da48aa9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:11.047348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 111/233] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:10 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:11.059335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54844b0c-e452-49d0-a354-561d99a0548d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:11.172729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:11.179849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54844b0c-e452-49d0-a354-561d99a0548d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:11.215648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f247394-f8cb-41c0-8987-25abff5321d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:11.216565 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f247394-f8cb-41c0-8987-25abff5321d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:11.217013 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f247394-f8cb-41c0-8987-25abff5321d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:11.217424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f247394-f8cb-41c0-8987-25abff5321d5 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/users/bc2dc60490bc466c845c20f678cf0d87/roles/b69a8899fce14a0d839745bb76b0f441` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:11.219704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f247394-f8cb-41c0-8987-25abff5321d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3, user_id=bc2dc60490bc466c845c20f678cf0d87, role_id=b69a8899fce14a0d839745bb76b0f441)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:11.221306 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f247394-f8cb-41c0-8987-25abff5321d5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:11.326368 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 123/234] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:19:11 2022] PUT /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/users/bc2dc60490bc466c845c20f678cf0d87/roles/b69a8899fce14a0d839745bb76b0f441 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:19:11.778122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947c521-917e-4938-9709-b4db3948c630 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:11.778635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947c521-917e-4938-9709-b4db3948c630 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:11.779051 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947c521-917e-4938-9709-b4db3948c630 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:11.780657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 112/235] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:11.787130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d576-0fee-4d23-b16f-9f163be0fc15 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:11.787665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d576-0fee-4d23-b16f-9f163be0fc15 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:11.788172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d576-0fee-4d23-b16f-9f163be0fc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:11.807483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cba7d576-0fee-4d23-b16f-9f163be0fc15 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:11.810590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cba7d576-0fee-4d23-b16f-9f163be0fc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:11.812349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2730cf-ec7a-41e7-af28-48c567d3f70d None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:11.812631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2730cf-ec7a-41e7-af28-48c567d3f70d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:11.812906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2730cf-ec7a-41e7-af28-48c567d3f70d None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:11.815354 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 113/236] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:11.825553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9ac2d-5aca-4d7e-b784-e98a55e81a03 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:11.825828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9ac2d-5aca-4d7e-b784-e98a55e81a03 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:11.826101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9ac2d-5aca-4d7e-b784-e98a55e81a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:11.853755 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-cfd9ac2d-5aca-4d7e-b784-e98a55e81a03 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:11.855753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cfd9ac2d-5aca-4d7e-b784-e98a55e81a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:11.930406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 124/237] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:11.950406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 114/238] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:11.991694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9ac2d-5aca-4d7e-b784-e98a55e81a03 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:11.991886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7d576-0fee-4d23-b16f-9f163be0fc15 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:12.118168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:12.136119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7d576-0fee-4d23-b16f-9f163be0fc15 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:12.174106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a39762-dcc7-45e3-aac1-5e9493cb90db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:12.175082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a39762-dcc7-45e3-aac1-5e9493cb90db None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:12.175533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a39762-dcc7-45e3-aac1-5e9493cb90db None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:12.175976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a39762-dcc7-45e3-aac1-5e9493cb90db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:12.177370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a39762-dcc7-45e3-aac1-5e9493cb90db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:12.178981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a39762-dcc7-45e3-aac1-5e9493cb90db None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:12.181094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 125/239] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:12.188934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a39762-dcc7-45e3-aac1-5e9493cb90db None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:12.189691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:12.200627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9ac2d-5aca-4d7e-b784-e98a55e81a03 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:12.264256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90096be4-851a-416e-af0a-4f195d9b562e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:12.264891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90096be4-851a-416e-af0a-4f195d9b562e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:12.265168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90096be4-851a-416e-af0a-4f195d9b562e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:12.265448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90096be4-851a-416e-af0a-4f195d9b562e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:12.288717 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90096be4-851a-416e-af0a-4f195d9b562e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-90096be4-851a-416e-af0a-4f195d9b562e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:12.294583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:12.295629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:12.296625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:12.298785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 115/240] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:12.304705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90096be4-851a-416e-af0a-4f195d9b562e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:12.354164 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a39762-dcc7-45e3-aac1-5e9493cb90db None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:12.391050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badc56d-2b88-475a-89d9-bf4b285348e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:12.391878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badc56d-2b88-475a-89d9-bf4b285348e2 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:12.392320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badc56d-2b88-475a-89d9-bf4b285348e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:12.393049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badc56d-2b88-475a-89d9-bf4b285348e2 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:12.394045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badc56d-2b88-475a-89d9-bf4b285348e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:12.395714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badc56d-2b88-475a-89d9-bf4b285348e2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:12.473155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90096be4-851a-416e-af0a-4f195d9b562e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:12.501427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 126/241] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:19:12 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:19:12.530103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1db55-6a30-4358-aed4-184b061ee861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:12.530962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1db55-6a30-4358-aed4-184b061ee861 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:12.531287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1db55-6a30-4358-aed4-184b061ee861 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:12.531571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1db55-6a30-4358-aed4-184b061ee861 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:12.532277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1db55-6a30-4358-aed4-184b061ee861 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:12.532627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1db55-6a30-4358-aed4-184b061ee861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:12.534076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1db55-6a30-4358-aed4-184b061ee861 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:12.544407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 116/242] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:12.553125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badc56d-2b88-475a-89d9-bf4b285348e2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:12.571821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:12.695637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badc56d-2b88-475a-89d9-bf4b285348e2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:12.743258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c5160-d037-41b2-87fd-6064caec8192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:12.744583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c5160-d037-41b2-87fd-6064caec8192 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:12.745632 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c5160-d037-41b2-87fd-6064caec8192 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:12.746114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c5160-d037-41b2-87fd-6064caec8192 None admin] PATH_INFO: `/v3/users/b38fc1b409fa401c972efad52b2ca127` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:12.767105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c5160-d037-41b2-87fd-6064caec8192 None admin] RBAC: Authorizing `identity:get_user(user_id=b38fc1b409fa401c972efad52b2ca127)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:12.769002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c5160-d037-41b2-87fd-6064caec8192 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:12.776645 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 127/243] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:12 2022] GET /identity/v3/users/b38fc1b409fa401c972efad52b2ca127 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:12.782282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1db55-6a30-4358-aed4-184b061ee861 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:12.806579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:12.901178 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1db55-6a30-4358-aed4-184b061ee861 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:12.936546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3eba02-6a3d-41e5-8061-f335e662312b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:12.937435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3eba02-6a3d-41e5-8061-f335e662312b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:12.937865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3eba02-6a3d-41e5-8061-f335e662312b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:12.938276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3eba02-6a3d-41e5-8061-f335e662312b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:12.944997 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df6316-d01d-49c4-b0f3-0c6875da7797 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:12.945271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df6316-d01d-49c4-b0f3-0c6875da7797 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:12.945866 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df6316-d01d-49c4-b0f3-0c6875da7797 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:12.949576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 128/244] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:12.954152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a9c49-d426-4cd2-9658-df15c519650e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:12.954670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a9c49-d426-4cd2-9658-df15c519650e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:12.955162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a9c49-d426-4cd2-9658-df15c519650e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:12.981430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-981a9c49-d426-4cd2-9658-df15c519650e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:12.983015 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-981a9c49-d426-4cd2-9658-df15c519650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:13.034245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3eba02-6a3d-41e5-8061-f335e662312b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:13.035942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3eba02-6a3d-41e5-8061-f335e662312b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:13.063139 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:13.073911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 129/245] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:13.074627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:13.097388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a9c49-d426-4cd2-9658-df15c519650e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:13.183373 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 117/246] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:13.190689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3eba02-6a3d-41e5-8061-f335e662312b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:13.219115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a9c49-d426-4cd2-9658-df15c519650e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:13.267107 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ea044-196b-40e9-96fb-3e57edb77dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:13.268147 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ea044-196b-40e9-96fb-3e57edb77dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:13.268628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ea044-196b-40e9-96fb-3e57edb77dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:13.269134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ea044-196b-40e9-96fb-3e57edb77dc5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:13.281539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ea044-196b-40e9-96fb-3e57edb77dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f08ea044-196b-40e9-96fb-3e57edb77dc5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:13.286618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:13.287491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:13.288330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:13.294318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 130/247] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:13.294318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5479bd-86f9-4b96-a1f5-ba6335a1b979 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:13.294318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5479bd-86f9-4b96-a1f5-ba6335a1b979 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:13.294318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5479bd-86f9-4b96-a1f5-ba6335a1b979 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:13.294318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 131/248] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:13.296886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5479bd-86f9-4b96-a1f5-ba6335a1b979 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:13.330915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3eba02-6a3d-41e5-8061-f335e662312b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:13.354074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:13.381676 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dfad6-c53f-437d-9faa-305843e3d1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:13.381879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:13.382316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dfad6-c53f-437d-9faa-305843e3d1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:13.382596 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dfad6-c53f-437d-9faa-305843e3d1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:13.382965 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dfad6-c53f-437d-9faa-305843e3d1e4 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:13.384088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dfad6-c53f-437d-9faa-305843e3d1e4 None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:13.387026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dfad6-c53f-437d-9faa-305843e3d1e4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:13.389625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 118/249] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:13 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:13.394190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da6b2e-ab02-4a6b-ba58-9f810783afa3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:13.394472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da6b2e-ab02-4a6b-ba58-9f810783afa3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:13.394750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da6b2e-ab02-4a6b-ba58-9f810783afa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:13.433763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d3da6b2e-ab02-4a6b-ba58-9f810783afa3 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:13.434971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d3da6b2e-ab02-4a6b-ba58-9f810783afa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:13.470139 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5479bd-86f9-4b96-a1f5-ba6335a1b979 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:13.530019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 119/250] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:13.530700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da6b2e-ab02-4a6b-ba58-9f810783afa3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:13.576763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e7c90-2e40-4640-b313-8307c86b9a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:13.577401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7c90-2e40-4640-b313-8307c86b9a74 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:13.577681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7c90-2e40-4640-b313-8307c86b9a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:13.577959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7c90-2e40-4640-b313-8307c86b9a74 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:13.578742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e7c90-2e40-4640-b313-8307c86b9a74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:13.579031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e7c90-2e40-4640-b313-8307c86b9a74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:13.580542 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e7c90-2e40-4640-b313-8307c86b9a74 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:13.590212 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 132/251] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:13.591148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e7c90-2e40-4640-b313-8307c86b9a74 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:13.721292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da6b2e-ab02-4a6b-ba58-9f810783afa3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:13.745768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:13.752477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e7c90-2e40-4640-b313-8307c86b9a74 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:13.786303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a11b1-23a1-47c6-993d-3d97ea54b3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:13.786917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a11b1-23a1-47c6-993d-3d97ea54b3cf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:13.787198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a11b1-23a1-47c6-993d-3d97ea54b3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:13.787469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a11b1-23a1-47c6-993d-3d97ea54b3cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:13.788233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a11b1-23a1-47c6-993d-3d97ea54b3cf None admin] RBAC: Adding query filter params (role.id=0f9e8507916f478c80987f7c7fd3c3e9, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3, user.id=b38fc1b409fa401c972efad52b2ca127) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:13.788543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a11b1-23a1-47c6-993d-3d97ea54b3cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:13.789985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a11b1-23a1-47c6-993d-3d97ea54b3cf None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:13.797145 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:13.798542 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 120/252] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:13 2022] GET /identity/v3/role_assignments?user.id=b38fc1b409fa401c972efad52b2ca127&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=0f9e8507916f478c80987f7c7fd3c3e9 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:13.799198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a11b1-23a1-47c6-993d-3d97ea54b3cf None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:13.818171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a6a86-ec49-4ba0-86fe-e29e5977953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:13.818814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a6a86-ec49-4ba0-86fe-e29e5977953d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:13.819100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a6a86-ec49-4ba0-86fe-e29e5977953d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:13.819373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a6a86-ec49-4ba0-86fe-e29e5977953d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:13.841231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a6a86-ec49-4ba0-86fe-e29e5977953d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-af2a6a86-ec49-4ba0-86fe-e29e5977953d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:13.846473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:13.847334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:13.848132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:13.851019 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 133/253] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:19:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:13.858673 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a6a86-ec49-4ba0-86fe-e29e5977953d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:14.012269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a11b1-23a1-47c6-993d-3d97ea54b3cf None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:14.057713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a6a86-ec49-4ba0-86fe-e29e5977953d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:14.063937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8ddf1-1b22-4eb3-97a5-9becc44b747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:14.064805 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8ddf1-1b22-4eb3-97a5-9becc44b747b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:14.065239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8ddf1-1b22-4eb3-97a5-9becc44b747b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:14.065652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8ddf1-1b22-4eb3-97a5-9becc44b747b None admin] PATH_INFO: `/v3/users/ca1046f934824e73af9e1f0e9415b4c9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:14.068009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8ddf1-1b22-4eb3-97a5-9becc44b747b None admin] RBAC: Authorizing `identity:get_user(user_id=ca1046f934824e73af9e1f0e9415b4c9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:14.069809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8ddf1-1b22-4eb3-97a5-9becc44b747b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:14.071666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 121/254] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:13 2022] GET /identity/v3/users/ca1046f934824e73af9e1f0e9415b4c9 => generated 297 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:14.073523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281de8ca-c6fc-4167-8cae-1568b5b47f73 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:14.074207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281de8ca-c6fc-4167-8cae-1568b5b47f73 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:14.074684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281de8ca-c6fc-4167-8cae-1568b5b47f73 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:14.076317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 122/255] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:14.077451 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281de8ca-c6fc-4167-8cae-1568b5b47f73 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:14.103935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab2e68-ad89-4a54-885a-effb1949393d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:14.105745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab2e68-ad89-4a54-885a-effb1949393d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:14.106122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab2e68-ad89-4a54-885a-effb1949393d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:14.106406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab2e68-ad89-4a54-885a-effb1949393d None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:14.107140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab2e68-ad89-4a54-885a-effb1949393d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:14.107456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab2e68-ad89-4a54-885a-effb1949393d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:14.114091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab2e68-ad89-4a54-885a-effb1949393d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:14.133362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 134/256] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:14.139982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163a2cd-c377-4c46-b679-840837eaa1eb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:14.140130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:14.141511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163a2cd-c377-4c46-b679-840837eaa1eb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:14.142787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163a2cd-c377-4c46-b679-840837eaa1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:14.174139 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f163a2cd-c377-4c46-b679-840837eaa1eb None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:14.175679 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f163a2cd-c377-4c46-b679-840837eaa1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:14.240697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281de8ca-c6fc-4167-8cae-1568b5b47f73 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:14.246818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:14.302986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 135/257] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:14.304453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163a2cd-c377-4c46-b679-840837eaa1eb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:14.306120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b9450-2bb5-4e32-b95e-b77b2d9b7f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:14.306972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b9450-2bb5-4e32-b95e-b77b2d9b7f03 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:14.308634 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b9450-2bb5-4e32-b95e-b77b2d9b7f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:14.309110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b9450-2bb5-4e32-b95e-b77b2d9b7f03 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:14.311855 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b9450-2bb5-4e32-b95e-b77b2d9b7f03 None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:14.313541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b9450-2bb5-4e32-b95e-b77b2d9b7f03 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:14.316212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 123/258] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:14 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:14.321716 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b9450-2bb5-4e32-b95e-b77b2d9b7f03 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:14.477299 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b9450-2bb5-4e32-b95e-b77b2d9b7f03 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:14.491506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:14.507920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163a2cd-c377-4c46-b679-840837eaa1eb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:14.520036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca359f-1fd7-42da-9ee7-4170c5eea3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:14.520036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca359f-1fd7-42da-9ee7-4170c5eea3f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:14.520036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca359f-1fd7-42da-9ee7-4170c5eea3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:14.520036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca359f-1fd7-42da-9ee7-4170c5eea3f2 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/users/ca1046f934824e73af9e1f0e9415b4c9/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:14.520789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca359f-1fd7-42da-9ee7-4170c5eea3f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3, user_id=ca1046f934824e73af9e1f0e9415b4c9, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:14.522361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca359f-1fd7-42da-9ee7-4170c5eea3f2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:14.553443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0a97a-12f5-4062-9e61-601327ff21f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:14.554413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0a97a-12f5-4062-9e61-601327ff21f7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:14.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0a97a-12f5-4062-9e61-601327ff21f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:14.555263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0a97a-12f5-4062-9e61-601327ff21f7 None admin] PATH_INFO: `/v3/users/bc2dc60490bc466c845c20f678cf0d87` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:14.556478 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0a97a-12f5-4062-9e61-601327ff21f7 None admin] RBAC: Authorizing `identity:get_user(user_id=bc2dc60490bc466c845c20f678cf0d87)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:14.558626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0a97a-12f5-4062-9e61-601327ff21f7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:14.560650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 136/259] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:14 2022] GET /identity/v3/users/bc2dc60490bc466c845c20f678cf0d87 => generated 283 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:14.561483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0a97a-12f5-4062-9e61-601327ff21f7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:14.592919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 124/260] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:19:14 2022] PUT /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/users/ca1046f934824e73af9e1f0e9415b4c9/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:19:14.593937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca359f-1fd7-42da-9ee7-4170c5eea3f2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:14.615875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:14.688773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0a97a-12f5-4062-9e61-601327ff21f7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:14.738900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acc191-ebf3-47a1-ad81-f69186a3663a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:14.739993 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acc191-ebf3-47a1-ad81-f69186a3663a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:14.740426 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acc191-ebf3-47a1-ad81-f69186a3663a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:14.740808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acc191-ebf3-47a1-ad81-f69186a3663a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:14.753019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca359f-1fd7-42da-9ee7-4170c5eea3f2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:14.761620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5acc191-ebf3-47a1-ad81-f69186a3663a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-d5acc191-ebf3-47a1-ad81-f69186a3663a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:14.770116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:14.771079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:14.771849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:14.772546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 137/261] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:19:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:14.779647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acc191-ebf3-47a1-ad81-f69186a3663a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:14.792391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c190694-dd5f-4143-b4e2-0810ff48ff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:14.793025 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c190694-dd5f-4143-b4e2-0810ff48ff0d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:14.793312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c190694-dd5f-4143-b4e2-0810ff48ff0d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:14.793721 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c190694-dd5f-4143-b4e2-0810ff48ff0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:14.840956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:14.870970 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c190694-dd5f-4143-b4e2-0810ff48ff0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:14.872526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c190694-dd5f-4143-b4e2-0810ff48ff0d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:14.962108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:14.967794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acc191-ebf3-47a1-ad81-f69186a3663a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:15.018227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf2ae2-8378-4641-949a-83f235282db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:15.018869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2ae2-8378-4641-949a-83f235282db8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:15.019315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2ae2-8378-4641-949a-83f235282db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:15.019515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 125/262] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:15.019740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2ae2-8378-4641-949a-83f235282db8 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:15.020589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf2ae2-8378-4641-949a-83f235282db8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:15.020764 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf2ae2-8378-4641-949a-83f235282db8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:15.022158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf2ae2-8378-4641-949a-83f235282db8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:15.035110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 138/263] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:15.037412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c190694-dd5f-4143-b4e2-0810ff48ff0d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:15.040908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf2ae2-8378-4641-949a-83f235282db8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:15.148737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:15.207261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c190694-dd5f-4143-b4e2-0810ff48ff0d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:15.239487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf2ae2-8378-4641-949a-83f235282db8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:15.265673 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576f91b-33ec-4082-9d10-1e8198e32b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:15.266320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576f91b-33ec-4082-9d10-1e8198e32b89 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:15.266608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576f91b-33ec-4082-9d10-1e8198e32b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:15.266897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576f91b-33ec-4082-9d10-1e8198e32b89 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:15.270701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576f91b-33ec-4082-9d10-1e8198e32b89 None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:15.272259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576f91b-33ec-4082-9d10-1e8198e32b89 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:15.274252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 126/264] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:15 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:15.276002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfc56c-1db0-42d7-97d5-8d917e0aedcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:15.277880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfc56c-1db0-42d7-97d5-8d917e0aedcf None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:15.278301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfc56c-1db0-42d7-97d5-8d917e0aedcf None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:15.278718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfc56c-1db0-42d7-97d5-8d917e0aedcf None admin] PATH_INFO: `/v3/users/bc2dc60490bc466c845c20f678cf0d87` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:15.280334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfc56c-1db0-42d7-97d5-8d917e0aedcf None admin] RBAC: Authorizing `identity:get_user(user_id=bc2dc60490bc466c845c20f678cf0d87)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:15.280727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576f91b-33ec-4082-9d10-1e8198e32b89 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:15.282092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfc56c-1db0-42d7-97d5-8d917e0aedcf None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:15.284080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 139/265] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:15 2022] GET /identity/v3/users/bc2dc60490bc466c845c20f678cf0d87 => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:15.289547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfc56c-1db0-42d7-97d5-8d917e0aedcf None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:15.321794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:15.460474 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfc56c-1db0-42d7-97d5-8d917e0aedcf None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:15.484883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576f91b-33ec-4082-9d10-1e8198e32b89 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:15.537248 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86def7-2322-4460-a7e8-e0046ac66ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:15.538116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86def7-2322-4460-a7e8-e0046ac66ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:15.538562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86def7-2322-4460-a7e8-e0046ac66ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:15.538978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86def7-2322-4460-a7e8-e0046ac66ac7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:15.559697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f143-b632-4277-be48-b90333d77c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:15.560610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f143-b632-4277-be48-b90333d77c61 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:15.561032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f143-b632-4277-be48-b90333d77c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:15.561468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f143-b632-4277-be48-b90333d77c61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:15.562347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5f143-b632-4277-be48-b90333d77c61 None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3, user.id=bc2dc60490bc466c845c20f678cf0d87) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:15.562799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5f143-b632-4277-be48-b90333d77c61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:15.564403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5f143-b632-4277-be48-b90333d77c61 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:15.581634 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 127/266] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:15 2022] GET /identity/v3/role_assignments?user.id=bc2dc60490bc466c845c20f678cf0d87&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=108664305d2546949f7327b65d4cb667 => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:15.634506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86def7-2322-4460-a7e8-e0046ac66ac7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:15.636398 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86def7-2322-4460-a7e8-e0046ac66ac7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:15.667174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:15.824897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 140/267] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 22:19:15.830498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f143-b632-4277-be48-b90333d77c61 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:15.834508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:15.959425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f143-b632-4277-be48-b90333d77c61 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:15.985912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70dc9-19c7-4d67-a4d0-848f7402b3ee None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:15.986210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70dc9-19c7-4d67-a4d0-848f7402b3ee None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:15.986468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70dc9-19c7-4d67-a4d0-848f7402b3ee None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:15.987949 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 141/268] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:16.001110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e20832-aeab-4e05-ad67-428b67e56222 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.001383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e20832-aeab-4e05-ad67-428b67e56222 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.001837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e20832-aeab-4e05-ad67-428b67e56222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.018380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a65766-6cd4-4f92-9955-5513ce83082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:16.019325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a65766-6cd4-4f92-9955-5513ce83082c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.019757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a65766-6cd4-4f92-9955-5513ce83082c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.020204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a65766-6cd4-4f92-9955-5513ce83082c None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.023742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a65766-6cd4-4f92-9955-5513ce83082c None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:16.025604 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a65766-6cd4-4f92-9955-5513ce83082c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:16.029214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 128/269] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:15 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:16.037942 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-13e20832-aeab-4e05-ad67-428b67e56222 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:16.039363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-13e20832-aeab-4e05-ad67-428b67e56222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:16.053925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a65766-6cd4-4f92-9955-5513ce83082c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:16.141634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 142/270] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:16.160001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76381e-4e07-498b-91ee-eda47284fcd9 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.160512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76381e-4e07-498b-91ee-eda47284fcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.160929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76381e-4e07-498b-91ee-eda47284fcd9 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.162711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 143/271] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:16.169281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17f0ef-d353-4959-937b-d690ec275598 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.169746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17f0ef-d353-4959-937b-d690ec275598 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.170181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17f0ef-d353-4959-937b-d690ec275598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.191667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7b17f0ef-d353-4959-937b-d690ec275598 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:16.193913 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7b17f0ef-d353-4959-937b-d690ec275598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:16.198953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:16.248414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a65766-6cd4-4f92-9955-5513ce83082c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:16.272293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:16.290751 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 144/272] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:16.296042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17f0ef-d353-4959-937b-d690ec275598 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:16.303162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe71eb-44bb-458f-af3e-042fcd45e458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:16.304506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe71eb-44bb-458f-af3e-042fcd45e458 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.305018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe71eb-44bb-458f-af3e-042fcd45e458 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.305494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe71eb-44bb-458f-af3e-042fcd45e458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.306440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe71eb-44bb-458f-af3e-042fcd45e458 None admin] RBAC: Adding query filter params (role.id=b69a8899fce14a0d839745bb76b0f441, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3, user.id=bc2dc60490bc466c845c20f678cf0d87) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:16.307048 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe71eb-44bb-458f-af3e-042fcd45e458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:16.309090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe71eb-44bb-458f-af3e-042fcd45e458 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:16.326821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 129/273] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:16 2022] GET /identity/v3/role_assignments?user.id=bc2dc60490bc466c845c20f678cf0d87&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=b69a8899fce14a0d839745bb76b0f441 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:16.335539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe71eb-44bb-458f-af3e-042fcd45e458 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:16.480774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17f0ef-d353-4959-937b-d690ec275598 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:16.535627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733a175-3fab-4ebf-918d-5d56d63b1e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:16.536467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733a175-3fab-4ebf-918d-5d56d63b1e27 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.536742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733a175-3fab-4ebf-918d-5d56d63b1e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.537194 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733a175-3fab-4ebf-918d-5d56d63b1e27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.545210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe71eb-44bb-458f-af3e-042fcd45e458 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:16.549675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733a175-3fab-4ebf-918d-5d56d63b1e27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-1733a175-3fab-4ebf-918d-5d56d63b1e27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:16.554836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:16.555901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:16.556904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:16.559748 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 145/274] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:16.566828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733a175-3fab-4ebf-918d-5d56d63b1e27 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:16.585017 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8ab5e-2734-48b2-98de-a9615b01adda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:16.585639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8ab5e-2734-48b2-98de-a9615b01adda None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.586102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8ab5e-2734-48b2-98de-a9615b01adda None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.586520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8ab5e-2734-48b2-98de-a9615b01adda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.587780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8ab5e-2734-48b2-98de-a9615b01adda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:16.589942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8ab5e-2734-48b2-98de-a9615b01adda None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:16.592169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 130/275] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:16.597776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8ab5e-2734-48b2-98de-a9615b01adda None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:16.605230 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:16.623832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:16.681024 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733a175-3fab-4ebf-918d-5d56d63b1e27 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:16.715441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8ab5e-2734-48b2-98de-a9615b01adda None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:16.720483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fc433-eb89-42dd-a042-4a929035371d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:16.721773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fc433-eb89-42dd-a042-4a929035371d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.722449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fc433-eb89-42dd-a042-4a929035371d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.722917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fc433-eb89-42dd-a042-4a929035371d None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.724739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fc433-eb89-42dd-a042-4a929035371d None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:16.725307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fc433-eb89-42dd-a042-4a929035371d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:16.728929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fc433-eb89-42dd-a042-4a929035371d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:16.739284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 146/276] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:16.740449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906f04a-f534-420f-a2fb-b0a710f74bcf None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.740958 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906f04a-f534-420f-a2fb-b0a710f74bcf None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.741431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906f04a-f534-420f-a2fb-b0a710f74bcf None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.742947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 147/277] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:16.749267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906f04a-f534-420f-a2fb-b0a710f74bcf None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:16.757982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832f3af-d9ad-46a7-9874-d4c32e9b923d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:16.758594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f3af-d9ad-46a7-9874-d4c32e9b923d None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.758873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f3af-d9ad-46a7-9874-d4c32e9b923d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.759147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f3af-d9ad-46a7-9874-d4c32e9b923d None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.759796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832f3af-d9ad-46a7-9874-d4c32e9b923d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:16.761462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832f3af-d9ad-46a7-9874-d4c32e9b923d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:16.851428 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906f04a-f534-420f-a2fb-b0a710f74bcf None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:16.862755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:16.875137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 131/278] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:19:16 2022] POST /identity/v3/users => generated 293 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:19:16.876674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e170a47-d5ee-4f4a-bfb4-07940243079e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.877127 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e170a47-d5ee-4f4a-bfb4-07940243079e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.877568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e170a47-d5ee-4f4a-bfb4-07940243079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.898439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8e170a47-d5ee-4f4a-bfb4-07940243079e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:16.899757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8e25a-34c6-4af5-9ea3-8aedbb5181ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:16.900331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8e170a47-d5ee-4f4a-bfb4-07940243079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:16.900529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8e25a-34c6-4af5-9ea3-8aedbb5181ad None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:16.900859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8e25a-34c6-4af5-9ea3-8aedbb5181ad None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:16.901164 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8e25a-34c6-4af5-9ea3-8aedbb5181ad None admin] PATH_INFO: `/v3/users/6ce0f1be16e94ce99f4b77f93afce80b` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:16.924424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca8e25a-34c6-4af5-9ea3-8aedbb5181ad None admin] RBAC: Authorizing `identity:get_user(user_id=6ce0f1be16e94ce99f4b77f93afce80b)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:16.926171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca8e25a-34c6-4af5-9ea3-8aedbb5181ad None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:16.933190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 148/279] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:16 2022] GET /identity/v3/users/6ce0f1be16e94ce99f4b77f93afce80b => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:16.936779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8e25a-34c6-4af5-9ea3-8aedbb5181ad None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:16.992172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:17.007976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 132/280] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:17.009638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7ff74-19ac-45ab-af4b-8b680a2efe81 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:17.010018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7ff74-19ac-45ab-af4b-8b680a2efe81 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:17.010293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7ff74-19ac-45ab-af4b-8b680a2efe81 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:17.012163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 133/281] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:17.022131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8748ba0-6717-40c0-beeb-a34402901b88 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:17.022459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8748ba0-6717-40c0-beeb-a34402901b88 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:17.022681 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8748ba0-6717-40c0-beeb-a34402901b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:17.043088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f8748ba0-6717-40c0-beeb-a34402901b88 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:17.045474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f8748ba0-6717-40c0-beeb-a34402901b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:17.078253 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8e25a-34c6-4af5-9ea3-8aedbb5181ad None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:17.119334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 134/282] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:17.120880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8748ba0-6717-40c0-beeb-a34402901b88 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:17.126541 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194d1d3-9679-4472-8434-33cd1375faf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:17.127268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194d1d3-9679-4472-8434-33cd1375faf3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:17.127457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194d1d3-9679-4472-8434-33cd1375faf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:17.127724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194d1d3-9679-4472-8434-33cd1375faf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:17.128565 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194d1d3-9679-4472-8434-33cd1375faf3 None admin] RBAC: Adding query filter params (role.id=87ab4761555e43419d0f23a0f31114f3, user.id=6ce0f1be16e94ce99f4b77f93afce80b, scope.system=all) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:17.128873 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194d1d3-9679-4472-8434-33cd1375faf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:17.130313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194d1d3-9679-4472-8434-33cd1375faf3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:17.138494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 149/283] 10.222.0.30 () {56 vars in 1308 bytes} [Sun Aug 7 22:19:16 2022] GET /identity/v3/role_assignments?user.id=6ce0f1be16e94ce99f4b77f93afce80b&scope.system=all&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:17.164256 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194d1d3-9679-4472-8434-33cd1375faf3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:17.205676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:17.254441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8748ba0-6717-40c0-beeb-a34402901b88 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:17.280078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:17.295556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595028fb-c788-4e86-972e-48c4ab4aa442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:17.296236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595028fb-c788-4e86-972e-48c4ab4aa442 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:17.296511 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595028fb-c788-4e86-972e-48c4ab4aa442 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:17.296776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595028fb-c788-4e86-972e-48c4ab4aa442 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:17.300499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194d1d3-9679-4472-8434-33cd1375faf3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:17.311758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595028fb-c788-4e86-972e-48c4ab4aa442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-595028fb-c788-4e86-972e-48c4ab4aa442 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:17.316930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:17.317834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:17.318624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 135/284] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:17.325256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595028fb-c788-4e86-972e-48c4ab4aa442 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:17.348883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391d217-bbf0-4fe8-b5dc-8524b6e98f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:17.349110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391d217-bbf0-4fe8-b5dc-8524b6e98f35 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:17.349466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391d217-bbf0-4fe8-b5dc-8524b6e98f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:17.349667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391d217-bbf0-4fe8-b5dc-8524b6e98f35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:17.361882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391d217-bbf0-4fe8-b5dc-8524b6e98f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-9391d217-bbf0-4fe8-b5dc-8524b6e98f35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:17.367602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:17.368512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:17.369284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 150/285] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:17.376434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391d217-bbf0-4fe8-b5dc-8524b6e98f35 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:17.452838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595028fb-c788-4e86-972e-48c4ab4aa442 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:17.517255 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:17.519910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe031b0-e831-4a5d-915c-c84c69e596d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:17.520556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe031b0-e831-4a5d-915c-c84c69e596d5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:17.520833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe031b0-e831-4a5d-915c-c84c69e596d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:17.521102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe031b0-e831-4a5d-915c-c84c69e596d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:17.521773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe031b0-e831-4a5d-915c-c84c69e596d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:17.522073 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe031b0-e831-4a5d-915c-c84c69e596d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:17.523441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe031b0-e831-4a5d-915c-c84c69e596d5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:17.529282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391d217-bbf0-4fe8-b5dc-8524b6e98f35 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:17.543981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 136/286] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:17.557109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe031b0-e831-4a5d-915c-c84c69e596d5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:17.566645 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f6b6c-4001-449f-b150-a9cdf369d75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:17.567644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f6b6c-4001-449f-b150-a9cdf369d75c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:17.568147 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f6b6c-4001-449f-b150-a9cdf369d75c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:17.568616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f6b6c-4001-449f-b150-a9cdf369d75c None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:17.569476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f6b6c-4001-449f-b150-a9cdf369d75c None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:17.569977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f6b6c-4001-449f-b150-a9cdf369d75c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:17.571560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f6b6c-4001-449f-b150-a9cdf369d75c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:17.581788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 151/287] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:17.587281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f6b6c-4001-449f-b150-a9cdf369d75c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:17.689932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:17.703911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f6b6c-4001-449f-b150-a9cdf369d75c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:17.745432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fccf3-41c4-47b4-9ba2-a2e992a1dd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:17.747458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fccf3-41c4-47b4-9ba2-a2e992a1dd1e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:17.747917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fccf3-41c4-47b4-9ba2-a2e992a1dd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:17.748419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fccf3-41c4-47b4-9ba2-a2e992a1dd1e None admin] PATH_INFO: `/v3/users/ca1046f934824e73af9e1f0e9415b4c9` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:17.749802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fccf3-41c4-47b4-9ba2-a2e992a1dd1e None admin] RBAC: Authorizing `identity:get_user(user_id=ca1046f934824e73af9e1f0e9415b4c9)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:17.751457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fccf3-41c4-47b4-9ba2-a2e992a1dd1e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:17.754125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 152/288] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:17 2022] GET /identity/v3/users/ca1046f934824e73af9e1f0e9415b4c9 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:17.765068 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fccf3-41c4-47b4-9ba2-a2e992a1dd1e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:17.800242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe031b0-e831-4a5d-915c-c84c69e596d5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:17.816376 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:17.863820 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88a29a-efe0-4754-b4c4-ecab2a8bf0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:17.864506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88a29a-efe0-4754-b4c4-ecab2a8bf0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:17.864783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88a29a-efe0-4754-b4c4-ecab2a8bf0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:17.865054 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88a29a-efe0-4754-b4c4-ecab2a8bf0d0 None admin] PATH_INFO: `/v3/users/b38fc1b409fa401c972efad52b2ca127` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:17.866187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88a29a-efe0-4754-b4c4-ecab2a8bf0d0 None admin] RBAC: Authorizing `identity:get_user(user_id=b38fc1b409fa401c972efad52b2ca127)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:17.867742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88a29a-efe0-4754-b4c4-ecab2a8bf0d0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:17.869352 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 137/289] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:17 2022] GET /identity/v3/users/b38fc1b409fa401c972efad52b2ca127 => generated 297 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:17.875566 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88a29a-efe0-4754-b4c4-ecab2a8bf0d0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:17.946005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fccf3-41c4-47b4-9ba2-a2e992a1dd1e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:18.012338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e001dd6-4913-477c-85cd-cfffca4452fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:18.012958 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e001dd6-4913-477c-85cd-cfffca4452fb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:18.013234 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e001dd6-4913-477c-85cd-cfffca4452fb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:18.013501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e001dd6-4913-477c-85cd-cfffca4452fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:18.071096 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88a29a-efe0-4754-b4c4-ecab2a8bf0d0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:18.091272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e001dd6-4913-477c-85cd-cfffca4452fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:18.093036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e001dd6-4913-477c-85cd-cfffca4452fb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:18.098221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:18.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:18.149480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6cadc-7595-4db3-b438-947940652745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:18.149976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6cadc-7595-4db3-b438-947940652745 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:18.150325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6cadc-7595-4db3-b438-947940652745 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:18.150611 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6cadc-7595-4db3-b438-947940652745 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:18.152415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6cadc-7595-4db3-b438-947940652745 None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:18.154116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6cadc-7595-4db3-b438-947940652745 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:18.155664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 138/290] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:17 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:18.161830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6cadc-7595-4db3-b438-947940652745 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:18.206871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 153/291] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:18.217796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e001dd6-4913-477c-85cd-cfffca4452fb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:18.337160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6cadc-7595-4db3-b438-947940652745 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:18.374154 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7212dfa-0634-41e0-abc7-0814413972d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:18.374823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7212dfa-0634-41e0-abc7-0814413972d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:18.375104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7212dfa-0634-41e0-abc7-0814413972d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:18.375381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7212dfa-0634-41e0-abc7-0814413972d7 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/users/b38fc1b409fa401c972efad52b2ca127/roles/0f9e8507916f478c80987f7c7fd3c3e9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:18.379224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7212dfa-0634-41e0-abc7-0814413972d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3, user_id=b38fc1b409fa401c972efad52b2ca127, role_id=0f9e8507916f478c80987f7c7fd3c3e9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:18.383076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7212dfa-0634-41e0-abc7-0814413972d7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:18.422085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e001dd6-4913-477c-85cd-cfffca4452fb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:18.446380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:18.458242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 139/292] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:19:18 2022] PUT /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/users/b38fc1b409fa401c972efad52b2ca127/roles/0f9e8507916f478c80987f7c7fd3c3e9 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:18.466316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffeea8-f035-4162-b1d6-ebe3d94aaba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:18.467312 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffeea8-f035-4162-b1d6-ebe3d94aaba5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:18.467797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffeea8-f035-4162-b1d6-ebe3d94aaba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:18.468283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffeea8-f035-4162-b1d6-ebe3d94aaba5 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:18.469765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffeea8-f035-4162-b1d6-ebe3d94aaba5 None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:18.471574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffeea8-f035-4162-b1d6-ebe3d94aaba5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:18.478763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 154/293] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:18 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:18.480169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7212dfa-0634-41e0-abc7-0814413972d7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:18.538047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:18.627031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7212dfa-0634-41e0-abc7-0814413972d7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:18.671148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da040021-3966-49fc-b539-2bba98f2cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:18.671767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da040021-3966-49fc-b539-2bba98f2cdfe None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:18.672075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da040021-3966-49fc-b539-2bba98f2cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:18.672352 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da040021-3966-49fc-b539-2bba98f2cdfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:18.673131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da040021-3966-49fc-b539-2bba98f2cdfe None admin] RBAC: Adding query filter params (role.id=87ab4761555e43419d0f23a0f31114f3, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3, user.id=ca1046f934824e73af9e1f0e9415b4c9) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:18.673441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da040021-3966-49fc-b539-2bba98f2cdfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:18.674929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da040021-3966-49fc-b539-2bba98f2cdfe None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:18.681445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 140/294] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:18 2022] GET /identity/v3/role_assignments?user.id=ca1046f934824e73af9e1f0e9415b4c9&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:19.342111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eafb06-217b-44dc-95d8-5747b4dff749 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:19.342790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eafb06-217b-44dc-95d8-5747b4dff749 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:19.343271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eafb06-217b-44dc-95d8-5747b4dff749 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:19.345104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 155/295] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:19.350200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7cd6d-b3f2-472c-b4df-fc63baf0fcf9 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:19.350634 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7cd6d-b3f2-472c-b4df-fc63baf0fcf9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:19.350799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7cd6d-b3f2-472c-b4df-fc63baf0fcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:19.369158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-95f7cd6d-b3f2-472c-b4df-fc63baf0fcf9 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:19.371214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-95f7cd6d-b3f2-472c-b4df-fc63baf0fcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:19.451013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 141/296] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:19.474476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eafb06-217b-44dc-95d8-5747b4dff749 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:19.515345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05e8e6-96e9-4037-8816-8cf0da64afbe None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:19.515972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05e8e6-96e9-4037-8816-8cf0da64afbe None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:19.516533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05e8e6-96e9-4037-8816-8cf0da64afbe None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:19.518115 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 142/297] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:19.525071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a83d37-33f2-4d7f-b39f-98b8e3fbaa12 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:19.525647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a83d37-33f2-4d7f-b39f-98b8e3fbaa12 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:19.526140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a83d37-33f2-4d7f-b39f-98b8e3fbaa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:19.546306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f6a83d37-33f2-4d7f-b39f-98b8e3fbaa12 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:19.547938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f6a83d37-33f2-4d7f-b39f-98b8e3fbaa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:19.576810 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:19.583563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eafb06-217b-44dc-95d8-5747b4dff749 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:19.619069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0225f-3d29-4de8-a63a-3506c5083e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:19.619971 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0225f-3d29-4de8-a63a-3506c5083e9e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:19.620312 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0225f-3d29-4de8-a63a-3506c5083e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:19.620634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0225f-3d29-4de8-a63a-3506c5083e9e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:19.623929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 143/298] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:19.632251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe820c-c726-49eb-b27b-5fb5f6f39a4e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:19.632251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe820c-c726-49eb-b27b-5fb5f6f39a4e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:19.632251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe820c-c726-49eb-b27b-5fb5f6f39a4e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:19.632251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 144/299] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:19.637336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9aee61-94dc-4e61-aaf2-621892d4f5be None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:19.637622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9aee61-94dc-4e61-aaf2-621892d4f5be None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:19.637897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9aee61-94dc-4e61-aaf2-621892d4f5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:19.640042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0225f-3d29-4de8-a63a-3506c5083e9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-39b0225f-3d29-4de8-a63a-3506c5083e9e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:19.644810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:19.645704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:19.646516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 156/300] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:19.652615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0225f-3d29-4de8-a63a-3506c5083e9e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:19.658624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1e9aee61-94dc-4e61-aaf2-621892d4f5be None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:19.659830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1e9aee61-94dc-4e61-aaf2-621892d4f5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:19.708331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:19.734812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 145/301] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:19.735794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9aee61-94dc-4e61-aaf2-621892d4f5be None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:19.772292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0225f-3d29-4de8-a63a-3506c5083e9e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:19.811141 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff591b84-a665-45d8-8699-2b2520daad90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:19.811767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff591b84-a665-45d8-8699-2b2520daad90 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:19.812063 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff591b84-a665-45d8-8699-2b2520daad90 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:19.812332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff591b84-a665-45d8-8699-2b2520daad90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:19.813458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff591b84-a665-45d8-8699-2b2520daad90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:19.814935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff591b84-a665-45d8-8699-2b2520daad90 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:19.816611 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 157/302] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:19.817405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff591b84-a665-45d8-8699-2b2520daad90 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:19.838282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:19.847760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9aee61-94dc-4e61-aaf2-621892d4f5be None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:19.884377 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be0f3c-1590-488a-bd36-fc76ba754e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:19.885055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be0f3c-1590-488a-bd36-fc76ba754e38 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:19.885330 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be0f3c-1590-488a-bd36-fc76ba754e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:19.886222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be0f3c-1590-488a-bd36-fc76ba754e38 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:19.886878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be0f3c-1590-488a-bd36-fc76ba754e38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:19.887182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be0f3c-1590-488a-bd36-fc76ba754e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:19.888580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be0f3c-1590-488a-bd36-fc76ba754e38 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:19.897495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 146/303] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:19.898275 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be0f3c-1590-488a-bd36-fc76ba754e38 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:19.929366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff591b84-a665-45d8-8699-2b2520daad90 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:19.948045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:19.964738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528e74f-f945-48e9-a64c-d465d67dbb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:19.965372 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528e74f-f945-48e9-a64c-d465d67dbb3a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:19.965652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528e74f-f945-48e9-a64c-d465d67dbb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:19.965934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528e74f-f945-48e9-a64c-d465d67dbb3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:19.982311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528e74f-f945-48e9-a64c-d465d67dbb3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-2528e74f-f945-48e9-a64c-d465d67dbb3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:19.987043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:19.987892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:19.988732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 158/304] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:19.989379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528e74f-f945-48e9-a64c-d465d67dbb3a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:20.017218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be0f3c-1590-488a-bd36-fc76ba754e38 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:20.052512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d9343-e13d-4cd1-ad95-deeb5bcdc16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:20.053118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d9343-e13d-4cd1-ad95-deeb5bcdc16d None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:20.053401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d9343-e13d-4cd1-ad95-deeb5bcdc16d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:20.053714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d9343-e13d-4cd1-ad95-deeb5bcdc16d None admin] PATH_INFO: `/v3/groups` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:20.054290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d9343-e13d-4cd1-ad95-deeb5bcdc16d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:20.055720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d9343-e13d-4cd1-ad95-deeb5bcdc16d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:20.096683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528e74f-f945-48e9-a64c-d465d67dbb3a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:20.136191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 147/305] 10.222.0.30 () {60 vars in 1134 bytes} [Sun Aug 7 22:19:19 2022] POST /identity/v3/groups => generated 229 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:19:20.136372 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07593d-8c98-42ef-9028-87224fc6d6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:20.136372 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07593d-8c98-42ef-9028-87224fc6d6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:20.136664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d9343-e13d-4cd1-ad95-deeb5bcdc16d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:20.136745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07593d-8c98-42ef-9028-87224fc6d6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:20.136745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07593d-8c98-42ef-9028-87224fc6d6c2 None admin] PATH_INFO: `/v3/users/63b78125f3da44e0b911dc77a68e4a3b` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:20.161290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:20.162688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07593d-8c98-42ef-9028-87224fc6d6c2 None admin] RBAC: Authorizing `identity:get_user(user_id=63b78125f3da44e0b911dc77a68e4a3b)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:20.164243 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07593d-8c98-42ef-9028-87224fc6d6c2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:20.166052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 159/306] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:19 2022] GET /identity/v3/users/63b78125f3da44e0b911dc77a68e4a3b => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:20.172877 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07593d-8c98-42ef-9028-87224fc6d6c2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:20.214010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:20.267660 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d9343-e13d-4cd1-ad95-deeb5bcdc16d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:20.292446 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07593d-8c98-42ef-9028-87224fc6d6c2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:20.308209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de4c75-999b-4b1e-9682-1f9072f71001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:20.309232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de4c75-999b-4b1e-9682-1f9072f71001 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:20.309657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de4c75-999b-4b1e-9682-1f9072f71001 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:20.310075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de4c75-999b-4b1e-9682-1f9072f71001 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:20.310931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de4c75-999b-4b1e-9682-1f9072f71001 None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:20.311482 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de4c75-999b-4b1e-9682-1f9072f71001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:20.313221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de4c75-999b-4b1e-9682-1f9072f71001 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:20.327130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 148/307] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:20.331328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bbb1f-31b7-4886-bc74-2aff460170a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:20.332175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bbb1f-31b7-4886-bc74-2aff460170a0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:20.332635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bbb1f-31b7-4886-bc74-2aff460170a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:20.333096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bbb1f-31b7-4886-bc74-2aff460170a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:20.333296 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de4c75-999b-4b1e-9682-1f9072f71001 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:20.333832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bbb1f-31b7-4886-bc74-2aff460170a0 None admin] RBAC: Adding query filter params (role.id=0f9e8507916f478c80987f7c7fd3c3e9, user.id=63b78125f3da44e0b911dc77a68e4a3b, scope.system=all) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:20.334144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bbb1f-31b7-4886-bc74-2aff460170a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:20.335582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bbb1f-31b7-4886-bc74-2aff460170a0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:20.342049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 160/308] 10.222.0.30 () {56 vars in 1308 bytes} [Sun Aug 7 22:19:20 2022] GET /identity/v3/role_assignments?user.id=63b78125f3da44e0b911dc77a68e4a3b&scope.system=all&role.id=0f9e8507916f478c80987f7c7fd3c3e9 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:20.428879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:20.455923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de4c75-999b-4b1e-9682-1f9072f71001 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:20.495908 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8965c-919f-4cee-bc6c-9be70bc835e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:20.496963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8965c-919f-4cee-bc6c-9be70bc835e5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:20.497243 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8965c-919f-4cee-bc6c-9be70bc835e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:20.497513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8965c-919f-4cee-bc6c-9be70bc835e5 None admin] PATH_INFO: `/v3/users/6ce0f1be16e94ce99f4b77f93afce80b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:20.498890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8965c-919f-4cee-bc6c-9be70bc835e5 None admin] RBAC: Authorizing `identity:get_user(user_id=6ce0f1be16e94ce99f4b77f93afce80b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:20.500340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8965c-919f-4cee-bc6c-9be70bc835e5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:20.502227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 149/309] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:20 2022] GET /identity/v3/users/6ce0f1be16e94ce99f4b77f93afce80b => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:20.507707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bbb1f-31b7-4886-bc74-2aff460170a0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:20.609020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:20.612713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bbb1f-31b7-4886-bc74-2aff460170a0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:20.648229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc2344-9a9d-4ca5-8c0e-6bc5cb165238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:20.648880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2344-9a9d-4ca5-8c0e-6bc5cb165238 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:20.649155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2344-9a9d-4ca5-8c0e-6bc5cb165238 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:20.649431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2344-9a9d-4ca5-8c0e-6bc5cb165238 None admin] PATH_INFO: `/v3/system/users/6ce0f1be16e94ce99f4b77f93afce80b/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:20.650879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc2344-9a9d-4ca5-8c0e-6bc5cb165238 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6ce0f1be16e94ce99f4b77f93afce80b, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:20.652483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc2344-9a9d-4ca5-8c0e-6bc5cb165238 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:20.737799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 161/310] 10.222.0.30 () {58 vars in 1334 bytes} [Sun Aug 7 22:19:20 2022] PUT /identity/v3/system/users/6ce0f1be16e94ce99f4b77f93afce80b/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:19:20.750644 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ff2b0-13e1-470b-a92f-358d54d50277 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:20.750942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ff2b0-13e1-470b-a92f-358d54d50277 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:20.751200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ff2b0-13e1-470b-a92f-358d54d50277 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:20.752943 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 150/311] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:20.758674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a025e-6b09-4e46-b6a5-752322eeefdb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:20.759008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a025e-6b09-4e46-b6a5-752322eeefdb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:20.759258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a025e-6b09-4e46-b6a5-752322eeefdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:20.780381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e48a025e-6b09-4e46-b6a5-752322eeefdb None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:20.781761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e48a025e-6b09-4e46-b6a5-752322eeefdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:20.857920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 162/312] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:20.881594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ff2b0-13e1-470b-a92f-358d54d50277 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:20.991592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ff2b0-13e1-470b-a92f-358d54d50277 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:21.028336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234e272-f8da-44b0-91f0-2af53e8b5aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:21.028974 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234e272-f8da-44b0-91f0-2af53e8b5aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:21.029254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234e272-f8da-44b0-91f0-2af53e8b5aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:21.029530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234e272-f8da-44b0-91f0-2af53e8b5aa3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:21.040255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234e272-f8da-44b0-91f0-2af53e8b5aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-8234e272-f8da-44b0-91f0-2af53e8b5aa3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:21.045111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:21.045982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:21.046944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 151/313] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:21.052270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a025e-6b09-4e46-b6a5-752322eeefdb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:21.154449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:21.161198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a025e-6b09-4e46-b6a5-752322eeefdb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:21.197695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321950b-a6ff-4dec-a68c-715214acf92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:21.198814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321950b-a6ff-4dec-a68c-715214acf92e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:21.199100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321950b-a6ff-4dec-a68c-715214acf92e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:21.199378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321950b-a6ff-4dec-a68c-715214acf92e None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:21.200083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321950b-a6ff-4dec-a68c-715214acf92e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:21.200397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321950b-a6ff-4dec-a68c-715214acf92e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:21.201797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321950b-a6ff-4dec-a68c-715214acf92e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:21.210566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 163/314] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:21.215968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234e272-f8da-44b0-91f0-2af53e8b5aa3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:21.220425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:21.322186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234e272-f8da-44b0-91f0-2af53e8b5aa3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:21.357777 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec776eff-8734-4ba6-9647-a618ed49b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:21.358783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec776eff-8734-4ba6-9647-a618ed49b335 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:21.359258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec776eff-8734-4ba6-9647-a618ed49b335 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:21.359770 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec776eff-8734-4ba6-9647-a618ed49b335 None admin] PATH_INFO: `/v3/users/b38fc1b409fa401c972efad52b2ca127` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:21.361126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec776eff-8734-4ba6-9647-a618ed49b335 None admin] RBAC: Authorizing `identity:get_user(user_id=b38fc1b409fa401c972efad52b2ca127)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:21.362757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec776eff-8734-4ba6-9647-a618ed49b335 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:21.364902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 152/315] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:21 2022] GET /identity/v3/users/b38fc1b409fa401c972efad52b2ca127 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:21.371320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321950b-a6ff-4dec-a68c-715214acf92e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:21.478937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321950b-a6ff-4dec-a68c-715214acf92e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:21.514938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4f07f-8807-4a7b-b979-e3bf0b85c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:21.515539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4f07f-8807-4a7b-b979-e3bf0b85c91e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:21.515804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4f07f-8807-4a7b-b979-e3bf0b85c91e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:21.516105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4f07f-8807-4a7b-b979-e3bf0b85c91e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:21.541282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:21.587002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4f07f-8807-4a7b-b979-e3bf0b85c91e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:21.588400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4f07f-8807-4a7b-b979-e3bf0b85c91e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:21.693653 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 164/316] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 22:19:21.699632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec776eff-8734-4ba6-9647-a618ed49b335 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:21.779160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:21.800533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec776eff-8734-4ba6-9647-a618ed49b335 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:21.835531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c2a75-2cd5-42d3-9d65-e4eb4c32d0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:21.836189 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c2a75-2cd5-42d3-9d65-e4eb4c32d0cf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:21.836466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c2a75-2cd5-42d3-9d65-e4eb4c32d0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:21.836739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c2a75-2cd5-42d3-9d65-e4eb4c32d0cf None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:21.837914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c2a75-2cd5-42d3-9d65-e4eb4c32d0cf None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:21.839877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c2a75-2cd5-42d3-9d65-e4eb4c32d0cf None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:21.841812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 153/317] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:21 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:21.847601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4f07f-8807-4a7b-b979-e3bf0b85c91e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:21.905661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:21.958914 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4f07f-8807-4a7b-b979-e3bf0b85c91e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:22.000511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec3235-bf6b-4a8e-a879-398cc0c30320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:22.001135 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3235-bf6b-4a8e-a879-398cc0c30320 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:22.001419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3235-bf6b-4a8e-a879-398cc0c30320 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:22.001697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3235-bf6b-4a8e-a879-398cc0c30320 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:22.002434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec3235-bf6b-4a8e-a879-398cc0c30320 None admin] RBAC: Adding query filter params (role.id=0f9e8507916f478c80987f7c7fd3c3e9, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3, user.id=b38fc1b409fa401c972efad52b2ca127) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:22.002749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec3235-bf6b-4a8e-a879-398cc0c30320 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:22.004235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec3235-bf6b-4a8e-a879-398cc0c30320 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:22.011319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 165/318] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:21 2022] GET /identity/v3/role_assignments?user.id=b38fc1b409fa401c972efad52b2ca127&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=0f9e8507916f478c80987f7c7fd3c3e9 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:22.348073 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebab3-74fc-4c47-a505-42b78bbef630 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:22.348444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebab3-74fc-4c47-a505-42b78bbef630 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:22.348642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebab3-74fc-4c47-a505-42b78bbef630 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:22.350100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 154/319] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:22.355024 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7931c-ccd3-4a46-8778-58466104bb8b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:22.355307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7931c-ccd3-4a46-8778-58466104bb8b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:22.355583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7931c-ccd3-4a46-8778-58466104bb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:22.374620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c3d7931c-ccd3-4a46-8778-58466104bb8b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:22.375800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c3d7931c-ccd3-4a46-8778-58466104bb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:22.446036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 166/320] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:22.468411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cebab3-74fc-4c47-a505-42b78bbef630 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:22.491529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd683db-001d-4c05-b7bd-0dd0f1560757 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:22.491856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd683db-001d-4c05-b7bd-0dd0f1560757 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:22.492642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd683db-001d-4c05-b7bd-0dd0f1560757 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:22.494119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 167/321] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:22.499753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e09c7-c6c0-4f78-b1bf-72b487578deb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:22.500052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e09c7-c6c0-4f78-b1bf-72b487578deb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:22.500325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e09c7-c6c0-4f78-b1bf-72b487578deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:22.519252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-965e09c7-c6c0-4f78-b1bf-72b487578deb None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:22.520459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-965e09c7-c6c0-4f78-b1bf-72b487578deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:22.569077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:22.580313 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cebab3-74fc-4c47-a505-42b78bbef630 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:22.591910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 168/322] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:22.614078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e09c7-c6c0-4f78-b1bf-72b487578deb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:22.615090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ab1ff-7aa7-4229-af20-56390b9bd875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:22.615878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ab1ff-7aa7-4229-af20-56390b9bd875 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:22.616324 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ab1ff-7aa7-4229-af20-56390b9bd875 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:22.616594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ab1ff-7aa7-4229-af20-56390b9bd875 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:22.617722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ab1ff-7aa7-4229-af20-56390b9bd875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:22.619166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ab1ff-7aa7-4229-af20-56390b9bd875 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:22.626801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 155/323] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:22.627573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ab1ff-7aa7-4229-af20-56390b9bd875 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:22.640075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:22.741148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ab1ff-7aa7-4229-af20-56390b9bd875 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:22.757829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e09c7-c6c0-4f78-b1bf-72b487578deb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:22.775666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49867ad6-eac1-4e7b-a37a-f096fb873489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:22.776447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49867ad6-eac1-4e7b-a37a-f096fb873489 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:22.776878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49867ad6-eac1-4e7b-a37a-f096fb873489 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:22.777157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49867ad6-eac1-4e7b-a37a-f096fb873489 None admin] PATH_INFO: `/v3/groups` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:22.777801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49867ad6-eac1-4e7b-a37a-f096fb873489 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:22.779255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49867ad6-eac1-4e7b-a37a-f096fb873489 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:22.803216 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60061b-2f16-4dc6-b22e-d577fa8ffce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:22.804208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60061b-2f16-4dc6-b22e-d577fa8ffce8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:22.804704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60061b-2f16-4dc6-b22e-d577fa8ffce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:22.805225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60061b-2f16-4dc6-b22e-d577fa8ffce8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:22.816865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60061b-2f16-4dc6-b22e-d577fa8ffce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-fb60061b-2f16-4dc6-b22e-d577fa8ffce8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:22.821909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:22.823383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:22.824301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:22.825373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 169/324] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:22.831171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60061b-2f16-4dc6-b22e-d577fa8ffce8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:22.860438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 156/325] 10.222.0.30 () {60 vars in 1134 bytes} [Sun Aug 7 22:19:22 2022] POST /identity/v3/groups => generated 226 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:19:22.878270 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980a5e8-c981-4ef6-b89e-eda20cec0637 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:22.878454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980a5e8-c981-4ef6-b89e-eda20cec0637 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:22.878818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980a5e8-c981-4ef6-b89e-eda20cec0637 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:22.880610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 157/326] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:22.886696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e620a-48fd-4767-9b46-73ee3e257daf None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:22.886979 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e620a-48fd-4767-9b46-73ee3e257daf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:22.887742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e620a-48fd-4767-9b46-73ee3e257daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:22.897458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:22.920273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c37e620a-48fd-4767-9b46-73ee3e257daf None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:22.922221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c37e620a-48fd-4767-9b46-73ee3e257daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:22.971659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60061b-2f16-4dc6-b22e-d577fa8ffce8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:23.020788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9329a66-6f8a-4cb1-9d93-12a178f80afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:23.021010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9329a66-6f8a-4cb1-9d93-12a178f80afd None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:23.021266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9329a66-6f8a-4cb1-9d93-12a178f80afd None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:23.021438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9329a66-6f8a-4cb1-9d93-12a178f80afd None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:23.022249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9329a66-6f8a-4cb1-9d93-12a178f80afd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:23.023291 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9329a66-6f8a-4cb1-9d93-12a178f80afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:23.023957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9329a66-6f8a-4cb1-9d93-12a178f80afd None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:23.025233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:23.032483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 158/327] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:23.035341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 170/328] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:23.043047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e620a-48fd-4767-9b46-73ee3e257daf None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:23.057200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9329a66-6f8a-4cb1-9d93-12a178f80afd None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:23.166692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e620a-48fd-4767-9b46-73ee3e257daf None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:23.187024 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9329a66-6f8a-4cb1-9d93-12a178f80afd None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:23.188593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:23.215094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403cd56-8c42-432a-8039-3d192ceb519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:23.215501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403cd56-8c42-432a-8039-3d192ceb519d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:23.215956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403cd56-8c42-432a-8039-3d192ceb519d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:23.216188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403cd56-8c42-432a-8039-3d192ceb519d None admin] PATH_INFO: `/v3/users/63b78125f3da44e0b911dc77a68e4a3b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:23.217394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403cd56-8c42-432a-8039-3d192ceb519d None admin] RBAC: Authorizing `identity:get_user(user_id=63b78125f3da44e0b911dc77a68e4a3b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:23.218841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403cd56-8c42-432a-8039-3d192ceb519d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:23.220741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 159/329] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:23 2022] GET /identity/v3/users/63b78125f3da44e0b911dc77a68e4a3b => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:23.226081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403cd56-8c42-432a-8039-3d192ceb519d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:23.234835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b150bc-0fd0-4c67-920e-8dc4e33eab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:23.235480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b150bc-0fd0-4c67-920e-8dc4e33eab19 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:23.236326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b150bc-0fd0-4c67-920e-8dc4e33eab19 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:23.236610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b150bc-0fd0-4c67-920e-8dc4e33eab19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:23.250346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b150bc-0fd0-4c67-920e-8dc4e33eab19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b5b150bc-0fd0-4c67-920e-8dc4e33eab19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:23.257487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:23.258350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:23.259157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 171/330] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:23.266429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b150bc-0fd0-4c67-920e-8dc4e33eab19 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:23.306194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:23.349814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403cd56-8c42-432a-8039-3d192ceb519d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:23.390023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661990-f9c4-4246-b17c-9d741ba3630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:23.390422 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b150bc-0fd0-4c67-920e-8dc4e33eab19 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:23.390715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661990-f9c4-4246-b17c-9d741ba3630b None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:23.390996 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661990-f9c4-4246-b17c-9d741ba3630b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:23.391271 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661990-f9c4-4246-b17c-9d741ba3630b None admin] PATH_INFO: `/v3/system/users/63b78125f3da44e0b911dc77a68e4a3b/roles/0f9e8507916f478c80987f7c7fd3c3e9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:23.393289 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13661990-f9c4-4246-b17c-9d741ba3630b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=63b78125f3da44e0b911dc77a68e4a3b, role_id=0f9e8507916f478c80987f7c7fd3c3e9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:23.394758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13661990-f9c4-4246-b17c-9d741ba3630b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:23.434242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fd47c-172f-48df-98d6-a1c87401bc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:23.435186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fd47c-172f-48df-98d6-a1c87401bc69 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:23.435668 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fd47c-172f-48df-98d6-a1c87401bc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:23.436003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fd47c-172f-48df-98d6-a1c87401bc69 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:23.436776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fd47c-172f-48df-98d6-a1c87401bc69 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:23.437169 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fd47c-172f-48df-98d6-a1c87401bc69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:23.438611 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fd47c-172f-48df-98d6-a1c87401bc69 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:23.449477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 172/331] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:23.454973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fd47c-172f-48df-98d6-a1c87401bc69 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:23.464282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 160/332] 10.222.0.30 () {58 vars in 1334 bytes} [Sun Aug 7 22:19:23 2022] PUT /identity/v3/system/users/63b78125f3da44e0b911dc77a68e4a3b/roles/0f9e8507916f478c80987f7c7fd3c3e9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:23.485281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:23.576911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fd47c-172f-48df-98d6-a1c87401bc69 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:23.618259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f75617-8a74-4af5-9b1b-afdae0a15205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:23.618960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f75617-8a74-4af5-9b1b-afdae0a15205 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:23.619240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f75617-8a74-4af5-9b1b-afdae0a15205 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:23.619791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f75617-8a74-4af5-9b1b-afdae0a15205 None admin] PATH_INFO: `/v3/users/6ce0f1be16e94ce99f4b77f93afce80b` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:23.621116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f75617-8a74-4af5-9b1b-afdae0a15205 None admin] RBAC: Authorizing `identity:get_user(user_id=6ce0f1be16e94ce99f4b77f93afce80b)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:23.622566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f75617-8a74-4af5-9b1b-afdae0a15205 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:23.624505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 173/333] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:23 2022] GET /identity/v3/users/6ce0f1be16e94ce99f4b77f93afce80b => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:23.631332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661990-f9c4-4246-b17c-9d741ba3630b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:23.748839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661990-f9c4-4246-b17c-9d741ba3630b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:23.780066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:23.786859 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a529de9-6670-436c-86fc-0e54154bffbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:23.789746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a529de9-6670-436c-86fc-0e54154bffbf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:23.789746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a529de9-6670-436c-86fc-0e54154bffbf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:23.789746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a529de9-6670-436c-86fc-0e54154bffbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:23.789746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a529de9-6670-436c-86fc-0e54154bffbf None admin] RBAC: Adding query filter params (role.id=87ab4761555e43419d0f23a0f31114f3, user.id=6ce0f1be16e94ce99f4b77f93afce80b, scope.system=all) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:23.789746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a529de9-6670-436c-86fc-0e54154bffbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:23.791331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a529de9-6670-436c-86fc-0e54154bffbf None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:23.808398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 161/334] 10.222.0.30 () {56 vars in 1308 bytes} [Sun Aug 7 22:19:23 2022] GET /identity/v3/role_assignments?user.id=6ce0f1be16e94ce99f4b77f93afce80b&scope.system=all&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:25.306536 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75d584-8fe9-4097-994c-4ac12a42931e None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:25.307260 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75d584-8fe9-4097-994c-4ac12a42931e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:25.307260 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75d584-8fe9-4097-994c-4ac12a42931e None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:25.308893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 174/335] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:25.313844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa87507-eb3b-4682-b5f2-6c8f92d5874d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:25.314283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa87507-eb3b-4682-b5f2-6c8f92d5874d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:25.314476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa87507-eb3b-4682-b5f2-6c8f92d5874d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:25.337116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0fa87507-eb3b-4682-b5f2-6c8f92d5874d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:25.338862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0fa87507-eb3b-4682-b5f2-6c8f92d5874d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:25.426277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 162/336] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:25.462554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75d584-8fe9-4097-994c-4ac12a42931e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:25.566716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:25.580740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75d584-8fe9-4097-994c-4ac12a42931e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:25.619212 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cb3f6-746c-4f5a-bcc1-29d6affba8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:25.621071 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cb3f6-746c-4f5a-bcc1-29d6affba8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:25.621576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cb3f6-746c-4f5a-bcc1-29d6affba8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:25.622051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cb3f6-746c-4f5a-bcc1-29d6affba8a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:25.639269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cb3f6-746c-4f5a-bcc1-29d6affba8a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-946cb3f6-746c-4f5a-bcc1-29d6affba8a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:25.644427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:25.645311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:25.646089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:25.648730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 175/337] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:25.655035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa87507-eb3b-4682-b5f2-6c8f92d5874d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:25.763980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b7ff-4e9f-4f13-8d11-68c2c4bfe917 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:25.764644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b7ff-4e9f-4f13-8d11-68c2c4bfe917 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:25.765124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b7ff-4e9f-4f13-8d11-68c2c4bfe917 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:25.767872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 176/338] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:25.773003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0e4e6-7a99-427c-8fc5-4bc30689e15b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:25.773540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0e4e6-7a99-427c-8fc5-4bc30689e15b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:25.774019 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0e4e6-7a99-427c-8fc5-4bc30689e15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:25.814486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa87507-eb3b-4682-b5f2-6c8f92d5874d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:25.815775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c2a0e4e6-7a99-427c-8fc5-4bc30689e15b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:25.817325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c2a0e4e6-7a99-427c-8fc5-4bc30689e15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:25.844371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:25.858913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ea442-ad66-43ed-8de1-d8ab1d7fed55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:25.859684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ea442-ad66-43ed-8de1-d8ab1d7fed55 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:25.861765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ea442-ad66-43ed-8de1-d8ab1d7fed55 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:25.861765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ea442-ad66-43ed-8de1-d8ab1d7fed55 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:25.861765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ea442-ad66-43ed-8de1-d8ab1d7fed55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:25.861765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ea442-ad66-43ed-8de1-d8ab1d7fed55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:25.862859 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ea442-ad66-43ed-8de1-d8ab1d7fed55 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:25.871851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 163/339] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:25.877328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ea442-ad66-43ed-8de1-d8ab1d7fed55 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:26.025411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 177/340] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:26.028367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ea442-ad66-43ed-8de1-d8ab1d7fed55 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:26.065495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0e4e6-7a99-427c-8fc5-4bc30689e15b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:26.076762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813e592-0e8f-4736-92f6-8e75622d38ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:26.077503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813e592-0e8f-4736-92f6-8e75622d38ac None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:26.077768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813e592-0e8f-4736-92f6-8e75622d38ac None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:26.078041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813e592-0e8f-4736-92f6-8e75622d38ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:26.144404 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:26.164795 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813e592-0e8f-4736-92f6-8e75622d38ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:26.166267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813e592-0e8f-4736-92f6-8e75622d38ac None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:26.218568 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:26.237053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0e4e6-7a99-427c-8fc5-4bc30689e15b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:26.304898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77c302-0e61-4cde-ad6b-165241b8e68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:26.305562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77c302-0e61-4cde-ad6b-165241b8e68c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:26.306041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77c302-0e61-4cde-ad6b-165241b8e68c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:26.306464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77c302-0e61-4cde-ad6b-165241b8e68c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:26.329185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77c302-0e61-4cde-ad6b-165241b8e68c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ee77c302-0e61-4cde-ad6b-165241b8e68c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:26.334582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:26.335656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:26.336672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:26.345524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 178/341] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:19:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:26.347087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78f224-74d6-43e9-9bf4-ff57a10a0ecd None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:26.347524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78f224-74d6-43e9-9bf4-ff57a10a0ecd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:26.347946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78f224-74d6-43e9-9bf4-ff57a10a0ecd None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:26.349437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 179/342] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:26.350830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78f224-74d6-43e9-9bf4-ff57a10a0ecd None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:26.366516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 164/343] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 22:19:26.368398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a597b83-fa44-42db-b4b5-33536acdef38 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:26.368834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a597b83-fa44-42db-b4b5-33536acdef38 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:26.369256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a597b83-fa44-42db-b4b5-33536acdef38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:26.407038 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8a597b83-fa44-42db-b4b5-33536acdef38 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:26.409882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8a597b83-fa44-42db-b4b5-33536acdef38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:26.564601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78f224-74d6-43e9-9bf4-ff57a10a0ecd None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:26.567177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 165/344] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:26.568036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a597b83-fa44-42db-b4b5-33536acdef38 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:26.597681 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:26.640287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ace8a-94ae-4673-8755-ec1b6811d35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:26.640957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ace8a-94ae-4673-8755-ec1b6811d35a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:26.641233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ace8a-94ae-4673-8755-ec1b6811d35a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:26.641501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ace8a-94ae-4673-8755-ec1b6811d35a None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:26.642172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ace8a-94ae-4673-8755-ec1b6811d35a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:26.642472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ace8a-94ae-4673-8755-ec1b6811d35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:26.643861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ace8a-94ae-4673-8755-ec1b6811d35a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:26.658091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 180/345] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:26.658091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ace8a-94ae-4673-8755-ec1b6811d35a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:26.660081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:26.729165 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a597b83-fa44-42db-b4b5-33536acdef38 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:26.771286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050ddc8-881a-40cb-8990-85d7e378f265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:26.771911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050ddc8-881a-40cb-8990-85d7e378f265 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:26.772221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050ddc8-881a-40cb-8990-85d7e378f265 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:26.772488 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050ddc8-881a-40cb-8990-85d7e378f265 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:26.774054 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050ddc8-881a-40cb-8990-85d7e378f265 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:26.777526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050ddc8-881a-40cb-8990-85d7e378f265 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:26.778713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 166/346] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:26 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:26.784268 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60472de6-385a-43cd-a8d9-28ca8311ab81 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:26.784789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60472de6-385a-43cd-a8d9-28ca8311ab81 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:26.786035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60472de6-385a-43cd-a8d9-28ca8311ab81 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:26.787666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 167/347] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:26.788811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60472de6-385a-43cd-a8d9-28ca8311ab81 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:26.815299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ace8a-94ae-4673-8755-ec1b6811d35a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:26.862124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbbae3-2b52-47f6-99a0-239d1454c4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:26.863094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbbae3-2b52-47f6-99a0-239d1454c4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:26.863584 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbbae3-2b52-47f6-99a0-239d1454c4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:26.864182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbbae3-2b52-47f6-99a0-239d1454c4f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:26.878528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdbbae3-2b52-47f6-99a0-239d1454c4f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-fcdbbae3-2b52-47f6-99a0-239d1454c4f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:26.884288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:26.885246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:26.886143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:26.888723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 181/348] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:26.889988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbbae3-2b52-47f6-99a0-239d1454c4f8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:26.890775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:26.922835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60472de6-385a-43cd-a8d9-28ca8311ab81 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:26.978579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:26.981156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edd594-2aa9-49e4-a3ff-12b209c179b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:26.982506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edd594-2aa9-49e4-a3ff-12b209c179b2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:26.982972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edd594-2aa9-49e4-a3ff-12b209c179b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:26.983421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edd594-2aa9-49e4-a3ff-12b209c179b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.027984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbbae3-2b52-47f6-99a0-239d1454c4f8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:27.075525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45feeb8-3257-4312-bda4-bcf53495afdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:27.076212 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45feeb8-3257-4312-bda4-bcf53495afdf None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:27.076498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45feeb8-3257-4312-bda4-bcf53495afdf None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:27.079797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45feeb8-3257-4312-bda4-bcf53495afdf None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.084214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45feeb8-3257-4312-bda4-bcf53495afdf None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:27.086563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45feeb8-3257-4312-bda4-bcf53495afdf None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:27.087548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edd594-2aa9-49e4-a3ff-12b209c179b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:27.090654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 182/349] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:26 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:27.090654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d66f0e-fd1d-4f06-a4fb-18e92a770451 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:27.090967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d66f0e-fd1d-4f06-a4fb-18e92a770451 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:27.091207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d66f0e-fd1d-4f06-a4fb-18e92a770451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.092840 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edd594-2aa9-49e4-a3ff-12b209c179b2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:27.113185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-92d66f0e-fd1d-4f06-a4fb-18e92a770451 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:27.115079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-92d66f0e-fd1d-4f06-a4fb-18e92a770451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:27.191969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:27.213349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 183/350] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:27.215806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d66f0e-fd1d-4f06-a4fb-18e92a770451 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:27.221321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 168/351] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:27.228142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edd594-2aa9-49e4-a3ff-12b209c179b2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:27.295265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:27.346853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d66f0e-fd1d-4f06-a4fb-18e92a770451 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:27.362535 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edd594-2aa9-49e4-a3ff-12b209c179b2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:27.393489 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296e544-53a1-4976-9773-d95927c5c228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:27.394048 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296e544-53a1-4976-9773-d95927c5c228 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:27.394331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296e544-53a1-4976-9773-d95927c5c228 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:27.394610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296e544-53a1-4976-9773-d95927c5c228 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.395359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296e544-53a1-4976-9773-d95927c5c228 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:27.395936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296e544-53a1-4976-9773-d95927c5c228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:27.397990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296e544-53a1-4976-9773-d95927c5c228 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:27.409221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa17c59-872f-44d4-9d09-767f23d2b0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:27.409744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 184/352] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:27.410438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296e544-53a1-4976-9773-d95927c5c228 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:27.411120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa17c59-872f-44d4-9d09-767f23d2b0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:27.411616 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa17c59-872f-44d4-9d09-767f23d2b0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:27.412129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa17c59-872f-44d4-9d09-767f23d2b0e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.413101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa17c59-872f-44d4-9d09-767f23d2b0e6 None admin] RBAC: Adding query filter params (group.id=405ae5a729404d04880e6c6b9ed925c3, role.id=87ab4761555e43419d0f23a0f31114f3, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:27.413608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa17c59-872f-44d4-9d09-767f23d2b0e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:27.415362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa17c59-872f-44d4-9d09-767f23d2b0e6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:27.422494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 169/353] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 22:19:27 2022] GET /identity/v3/role_assignments?group.id=405ae5a729404d04880e6c6b9ed925c3&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:27.423726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa17c59-872f-44d4-9d09-767f23d2b0e6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:27.480730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:27.552958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa17c59-872f-44d4-9d09-767f23d2b0e6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:27.562687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296e544-53a1-4976-9773-d95927c5c228 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:27.591071 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:27.603191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7965a1-1705-4287-a638-bbb646190e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:27.603917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7965a1-1705-4287-a638-bbb646190e43 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:27.604281 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7965a1-1705-4287-a638-bbb646190e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:27.604556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7965a1-1705-4287-a638-bbb646190e43 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.618832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e953-8c2f-4449-a91e-0a7909eb8781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:27.620151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e953-8c2f-4449-a91e-0a7909eb8781 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:27.620358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7965a1-1705-4287-a638-bbb646190e43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:27.620555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e953-8c2f-4449-a91e-0a7909eb8781 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:27.621050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e953-8c2f-4449-a91e-0a7909eb8781 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.622367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935e953-8c2f-4449-a91e-0a7909eb8781 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:27.623828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935e953-8c2f-4449-a91e-0a7909eb8781 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:27.625533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 185/354] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:27 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:27.626292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e953-8c2f-4449-a91e-0a7909eb8781 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-9c7965a1-1705-4287-a638-bbb646190e43 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:27.626773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:27.627662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 170/355] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:27.635588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136effac-8e70-46bf-8899-c11f68d9a273 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:27.636438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136effac-8e70-46bf-8899-c11f68d9a273 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:27.636438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136effac-8e70-46bf-8899-c11f68d9a273 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.636438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 171/356] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:27.636438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f379b-a529-4dda-b3c4-60ad0faa8a46 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:27.636438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f379b-a529-4dda-b3c4-60ad0faa8a46 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:27.636438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f379b-a529-4dda-b3c4-60ad0faa8a46 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.637081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 172/357] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:27.638400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f379b-a529-4dda-b3c4-60ad0faa8a46 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:27.762824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e953-8c2f-4449-a91e-0a7909eb8781 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:27.777279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f379b-a529-4dda-b3c4-60ad0faa8a46 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:27.779730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:27.810343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b82fac-23d3-4927-9d4d-3871036d5722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:27.810965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b82fac-23d3-4927-9d4d-3871036d5722 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:27.811239 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b82fac-23d3-4927-9d4d-3871036d5722 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:27.812663 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b82fac-23d3-4927-9d4d-3871036d5722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.831575 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d92fea-b53b-4885-b70a-3c5d9384d0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:27.832076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d92fea-b53b-4885-b70a-3c5d9384d0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:27.832484 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d92fea-b53b-4885-b70a-3c5d9384d0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:27.833716 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d92fea-b53b-4885-b70a-3c5d9384d0b7 None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.835599 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d92fea-b53b-4885-b70a-3c5d9384d0b7 None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:27.837060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d92fea-b53b-4885-b70a-3c5d9384d0b7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:27.839407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 173/358] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:27 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:27.844067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc0601-08c8-4174-956f-e2230404b509 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:27.844067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc0601-08c8-4174-956f-e2230404b509 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:27.844067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc0601-08c8-4174-956f-e2230404b509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:27.860211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0ffc0601-08c8-4174-956f-e2230404b509 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:27.861409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0ffc0601-08c8-4174-956f-e2230404b509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:27.880983 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:27.899170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b82fac-23d3-4927-9d4d-3871036d5722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:27.901590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b82fac-23d3-4927-9d4d-3871036d5722 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:27.949602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 174/359] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:27.950775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc0601-08c8-4174-956f-e2230404b509 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:28.039702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 186/360] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:28.039702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd1aa8-9632-4f33-89a6-54757445613d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:28.039702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd1aa8-9632-4f33-89a6-54757445613d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:28.039702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd1aa8-9632-4f33-89a6-54757445613d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:28.058003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-10fd1aa8-9632-4f33-89a6-54757445613d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:28.059503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-10fd1aa8-9632-4f33-89a6-54757445613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:28.087716 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:28.091184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc0601-08c8-4174-956f-e2230404b509 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:28.132623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c984a-5bf1-43ff-a780-1d9d43b54f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:28.133325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c984a-5bf1-43ff-a780-1d9d43b54f1a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:28.133607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c984a-5bf1-43ff-a780-1d9d43b54f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:28.133878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c984a-5bf1-43ff-a780-1d9d43b54f1a None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:28.134546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c984a-5bf1-43ff-a780-1d9d43b54f1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:28.134849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c984a-5bf1-43ff-a780-1d9d43b54f1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:28.137123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c984a-5bf1-43ff-a780-1d9d43b54f1a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:28.147896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 175/361] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:28.149155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c984a-5bf1-43ff-a780-1d9d43b54f1a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:28.151267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 187/362] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:28.152686 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd1aa8-9632-4f33-89a6-54757445613d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:28.178306 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:28.275653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c984a-5bf1-43ff-a780-1d9d43b54f1a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:28.297701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd1aa8-9632-4f33-89a6-54757445613d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:28.324462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c7c78-8273-4346-8a2f-a6d8bc9cb1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:28.325074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c7c78-8273-4346-8a2f-a6d8bc9cb1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:28.325356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c7c78-8273-4346-8a2f-a6d8bc9cb1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:28.325630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c7c78-8273-4346-8a2f-a6d8bc9cb1a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:28.326363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c7c78-8273-4346-8a2f-a6d8bc9cb1a0 None admin] RBAC: Adding query filter params (group.id=405ae5a729404d04880e6c6b9ed925c3, role.id=b69a8899fce14a0d839745bb76b0f441, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:28.326674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c7c78-8273-4346-8a2f-a6d8bc9cb1a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:28.328122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c7c78-8273-4346-8a2f-a6d8bc9cb1a0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:28.334638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 176/363] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 22:19:28 2022] GET /identity/v3/role_assignments?group.id=405ae5a729404d04880e6c6b9ed925c3&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=b69a8899fce14a0d839745bb76b0f441 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:28.335510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c7c78-8273-4346-8a2f-a6d8bc9cb1a0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:28.337860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8062b-1dd4-4777-893a-c6e4f1df20df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:28.339032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8062b-1dd4-4777-893a-c6e4f1df20df None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:28.339306 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8062b-1dd4-4777-893a-c6e4f1df20df None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:28.339576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8062b-1dd4-4777-893a-c6e4f1df20df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:28.353324 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8062b-1dd4-4777-893a-c6e4f1df20df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-17c8062b-1dd4-4777-893a-c6e4f1df20df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:28.358405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:28.359373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:28.360252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 188/364] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:19:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:28.360909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8062b-1dd4-4777-893a-c6e4f1df20df None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:28.377960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:28.458484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:28.468805 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c7c78-8273-4346-8a2f-a6d8bc9cb1a0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:28.485367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8062b-1dd4-4777-893a-c6e4f1df20df None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:28.517253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18ed1c-955a-41f1-bd85-3bc7c067710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:28.518055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18ed1c-955a-41f1-bd85-3bc7c067710f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:28.518336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18ed1c-955a-41f1-bd85-3bc7c067710f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:28.518923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18ed1c-955a-41f1-bd85-3bc7c067710f None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:28.521014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18ed1c-955a-41f1-bd85-3bc7c067710f None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:28.523140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18ed1c-955a-41f1-bd85-3bc7c067710f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:28.524941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 177/365] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:28 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:28.527701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddccc9-b222-4e93-ab1c-07850317726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:28.528372 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddccc9-b222-4e93-ab1c-07850317726c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:28.528649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddccc9-b222-4e93-ab1c-07850317726c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:28.528918 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddccc9-b222-4e93-ab1c-07850317726c None admin] PATH_INFO: `/v3/users/63b78125f3da44e0b911dc77a68e4a3b` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:28.530111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddccc9-b222-4e93-ab1c-07850317726c None admin] RBAC: Authorizing `identity:get_user(user_id=63b78125f3da44e0b911dc77a68e4a3b)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:28.530812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18ed1c-955a-41f1-bd85-3bc7c067710f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:28.531597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddccc9-b222-4e93-ab1c-07850317726c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:28.533415 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 189/366] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:19:28 2022] GET /identity/v3/users/63b78125f3da44e0b911dc77a68e4a3b => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:28.534435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddccc9-b222-4e93-ab1c-07850317726c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:28.666604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddccc9-b222-4e93-ab1c-07850317726c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:28.667238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18ed1c-955a-41f1-bd85-3bc7c067710f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:28.670037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:28.703332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c3d4c-799b-4ea9-80b4-1ca476a232ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:28.703956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3d4c-799b-4ea9-80b4-1ca476a232ae None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:28.704263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3d4c-799b-4ea9-80b4-1ca476a232ae None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:28.704542 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3d4c-799b-4ea9-80b4-1ca476a232ae None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:28.704653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a36fe-5f6f-41af-8de9-54a2d19156de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:28.705225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c3d4c-799b-4ea9-80b4-1ca476a232ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:28.705329 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a36fe-5f6f-41af-8de9-54a2d19156de None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:28.705527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a36fe-5f6f-41af-8de9-54a2d19156de None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:28.705619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c3d4c-799b-4ea9-80b4-1ca476a232ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:28.705800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a36fe-5f6f-41af-8de9-54a2d19156de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:28.706923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c3d4c-799b-4ea9-80b4-1ca476a232ae None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:28.716335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 190/367] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:28.717873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a36fe-5f6f-41af-8de9-54a2d19156de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:28.721807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c3d4c-799b-4ea9-80b4-1ca476a232ae None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ef5a36fe-5f6f-41af-8de9-54a2d19156de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:28.724265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:28.725176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:28.726045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 178/368] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:19:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:28.727188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a36fe-5f6f-41af-8de9-54a2d19156de None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:28.735289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:28.833900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c3d4c-799b-4ea9-80b4-1ca476a232ae None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:28.842503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a36fe-5f6f-41af-8de9-54a2d19156de None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:28.873000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f4bca-7eb9-4c2c-915e-2c953cab5c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:28.873641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f4bca-7eb9-4c2c-915e-2c953cab5c1d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:28.873922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f4bca-7eb9-4c2c-915e-2c953cab5c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:28.874257 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f4bca-7eb9-4c2c-915e-2c953cab5c1d None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:28.875401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f4bca-7eb9-4c2c-915e-2c953cab5c1d None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:28.877897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f4bca-7eb9-4c2c-915e-2c953cab5c1d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:28.879655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 191/369] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:28 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:28.880370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913def06-b183-4988-b7d8-b067b732fa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:28.880977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913def06-b183-4988-b7d8-b067b732fa90 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:28.881251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913def06-b183-4988-b7d8-b067b732fa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:28.881519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913def06-b183-4988-b7d8-b067b732fa90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:28.882245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913def06-b183-4988-b7d8-b067b732fa90 None admin] RBAC: Adding query filter params (role.id=0f9e8507916f478c80987f7c7fd3c3e9, user.id=63b78125f3da44e0b911dc77a68e4a3b, scope.system=all) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:28.882551 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913def06-b183-4988-b7d8-b067b732fa90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:28.883946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913def06-b183-4988-b7d8-b067b732fa90 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:28.884875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f4bca-7eb9-4c2c-915e-2c953cab5c1d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:28.897374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 179/370] 10.222.0.30 () {56 vars in 1308 bytes} [Sun Aug 7 22:19:28 2022] GET /identity/v3/role_assignments?user.id=63b78125f3da44e0b911dc77a68e4a3b&scope.system=all&role.id=0f9e8507916f478c80987f7c7fd3c3e9 => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:28.898686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913def06-b183-4988-b7d8-b067b732fa90 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:28.924545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:28.972383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:28.998524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f4bca-7eb9-4c2c-915e-2c953cab5c1d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:29.014964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913def06-b183-4988-b7d8-b067b732fa90 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:29.038530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65770a89-23ad-4bb7-b2a4-61709e756b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:29.039153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65770a89-23ad-4bb7-b2a4-61709e756b1d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:29.039434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65770a89-23ad-4bb7-b2a4-61709e756b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:29.039708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65770a89-23ad-4bb7-b2a4-61709e756b1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:29.054975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2e66c-ee56-40bb-900a-a57d27cd6c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:29.055590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2e66c-ee56-40bb-900a-a57d27cd6c1b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:29.055868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2e66c-ee56-40bb-900a-a57d27cd6c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:29.056194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2e66c-ee56-40bb-900a-a57d27cd6c1b None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:29.056861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2e66c-ee56-40bb-900a-a57d27cd6c1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:29.057471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2e66c-ee56-40bb-900a-a57d27cd6c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:29.059125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2e66c-ee56-40bb-900a-a57d27cd6c1b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:29.068973 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 180/371] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:29.069935 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2e66c-ee56-40bb-900a-a57d27cd6c1b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:29.118716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65770a89-23ad-4bb7-b2a4-61709e756b1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:29.120843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65770a89-23ad-4bb7-b2a4-61709e756b1d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:29.160413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:29.184476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2e66c-ee56-40bb-900a-a57d27cd6c1b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:29.205959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:29.223600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c627f8-4424-436e-818b-84248ea3f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:29.224772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c627f8-4424-436e-818b-84248ea3f1de None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:29.225183 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c627f8-4424-436e-818b-84248ea3f1de None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:29.225513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c627f8-4424-436e-818b-84248ea3f1de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:29.226419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c627f8-4424-436e-818b-84248ea3f1de None admin] RBAC: Adding query filter params (group.id=405ae5a729404d04880e6c6b9ed925c3, role.id=87ab4761555e43419d0f23a0f31114f3, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:29.226730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c627f8-4424-436e-818b-84248ea3f1de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:29.228709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c627f8-4424-436e-818b-84248ea3f1de None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:29.236388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 192/372] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:29.236556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 181/373] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 22:19:29 2022] GET /identity/v3/role_assignments?group.id=405ae5a729404d04880e6c6b9ed925c3&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:29.240277 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65770a89-23ad-4bb7-b2a4-61709e756b1d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:29.246317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c627f8-4424-436e-818b-84248ea3f1de None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:29.452545 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65770a89-23ad-4bb7-b2a4-61709e756b1d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:29.471704 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c627f8-4424-436e-818b-84248ea3f1de None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:29.520004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2495997-bb6f-4bda-95c0-13b0d87a25ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:29.520869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2495997-bb6f-4bda-95c0-13b0d87a25ce None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:29.521146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2495997-bb6f-4bda-95c0-13b0d87a25ce None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:29.521420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2495997-bb6f-4bda-95c0-13b0d87a25ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:29.535572 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923f391-08ea-4d40-8f46-37db434704ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:29.536590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f391-08ea-4d40-8f46-37db434704ae None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:29.537106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f391-08ea-4d40-8f46-37db434704ae None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:29.537931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f391-08ea-4d40-8f46-37db434704ae None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:29.539758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923f391-08ea-4d40-8f46-37db434704ae None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:29.542304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923f391-08ea-4d40-8f46-37db434704ae None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:29.545404 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 182/374] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:29 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:29.550705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923f391-08ea-4d40-8f46-37db434704ae None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:29.564106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:29.588566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:29.631201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2495997-bb6f-4bda-95c0-13b0d87a25ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:29.632709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2495997-bb6f-4bda-95c0-13b0d87a25ce None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:29.735989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923f391-08ea-4d40-8f46-37db434704ae None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:29.776406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace31851-b9e3-4309-af48-946224c6209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:29.777040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace31851-b9e3-4309-af48-946224c6209a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:29.777318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace31851-b9e3-4309-af48-946224c6209a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:29.777588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace31851-b9e3-4309-af48-946224c6209a None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:29.778710 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace31851-b9e3-4309-af48-946224c6209a None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:29.780198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace31851-b9e3-4309-af48-946224c6209a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:29.782052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 183/375] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:29 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:29.783354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 193/376] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:29.787637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace31851-b9e3-4309-af48-946224c6209a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:29.789380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2495997-bb6f-4bda-95c0-13b0d87a25ce None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:29.876417 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:29.901561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:29.902725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2495997-bb6f-4bda-95c0-13b0d87a25ce None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:29.949109 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914c28f-0989-46bb-a8f6-883cb2f9188f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:29.949750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c28f-0989-46bb-a8f6-883cb2f9188f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:29.950028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c28f-0989-46bb-a8f6-883cb2f9188f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:29.950301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c28f-0989-46bb-a8f6-883cb2f9188f None admin] PATH_INFO: `/v3/projects/ff949bd3888c436789a339dd0a2e16fc` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:29.951958 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914c28f-0989-46bb-a8f6-883cb2f9188f None admin] RBAC: Authorizing `identity:get_project(project_id=ff949bd3888c436789a339dd0a2e16fc)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:29.953423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914c28f-0989-46bb-a8f6-883cb2f9188f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:29.959888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 194/377] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:29 2022] GET /identity/v3/projects/ff949bd3888c436789a339dd0a2e16fc => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:29.959888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab370864-cc17-44a1-9ad4-a18eec0adffd None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:29.959888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab370864-cc17-44a1-9ad4-a18eec0adffd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:29.959888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab370864-cc17-44a1-9ad4-a18eec0adffd None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:29.959888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 195/378] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:29.962990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace31851-b9e3-4309-af48-946224c6209a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:29.965321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab370864-cc17-44a1-9ad4-a18eec0adffd None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:30.009554 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84887d05-032d-46db-9930-cc3ae83b78f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:30.010712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84887d05-032d-46db-9930-cc3ae83b78f7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:30.017374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84887d05-032d-46db-9930-cc3ae83b78f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:30.017374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84887d05-032d-46db-9930-cc3ae83b78f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:30.017374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84887d05-032d-46db-9930-cc3ae83b78f7 None admin] RBAC: Adding query filter params (group.id=405ae5a729404d04880e6c6b9ed925c3, role.id=b69a8899fce14a0d839745bb76b0f441, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:30.017374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84887d05-032d-46db-9930-cc3ae83b78f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:30.017374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84887d05-032d-46db-9930-cc3ae83b78f7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:30.022458 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 184/379] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 22:19:29 2022] GET /identity/v3/role_assignments?group.id=405ae5a729404d04880e6c6b9ed925c3&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=b69a8899fce14a0d839745bb76b0f441 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:30.023909 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113dc66-ee0f-45cf-9d85-91759e3c4716 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:30.025417 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113dc66-ee0f-45cf-9d85-91759e3c4716 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:30.026616 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113dc66-ee0f-45cf-9d85-91759e3c4716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:30.059692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d113dc66-ee0f-45cf-9d85-91759e3c4716 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:30.061279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d113dc66-ee0f-45cf-9d85-91759e3c4716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:30.085874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab370864-cc17-44a1-9ad4-a18eec0adffd None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:30.163826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752d033-513d-432c-b5a1-7a112a8d30ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:30.164510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752d033-513d-432c-b5a1-7a112a8d30ba None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:30.164993 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752d033-513d-432c-b5a1-7a112a8d30ba None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:30.165272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752d033-513d-432c-b5a1-7a112a8d30ba None admin] PATH_INFO: `/v3/groups/a68391f62e9c4c6bab2251c1627ec554` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:30.179475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752d033-513d-432c-b5a1-7a112a8d30ba None admin] RBAC: Authorizing `identity:get_group(group_id=a68391f62e9c4c6bab2251c1627ec554)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:30.180981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752d033-513d-432c-b5a1-7a112a8d30ba None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:30.183281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 196/380] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:29 2022] GET /identity/v3/groups/a68391f62e9c4c6bab2251c1627ec554 => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:30.189966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752d033-513d-432c-b5a1-7a112a8d30ba None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:30.208110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:30.210416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 185/381] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:30.247390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113dc66-ee0f-45cf-9d85-91759e3c4716 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:30.316273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752d033-513d-432c-b5a1-7a112a8d30ba None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:30.354294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:30.366567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b18e4d-0073-4997-a9db-5a63afe6f1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:30.367190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b18e4d-0073-4997-a9db-5a63afe6f1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:30.367467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b18e4d-0073-4997-a9db-5a63afe6f1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:30.367741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b18e4d-0073-4997-a9db-5a63afe6f1d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:30.368612 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b18e4d-0073-4997-a9db-5a63afe6f1d1 None admin] RBAC: Adding query filter params (group.id=a68391f62e9c4c6bab2251c1627ec554, role.id=108664305d2546949f7327b65d4cb667, scope.project.id=ff949bd3888c436789a339dd0a2e16fc) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:30.368921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b18e4d-0073-4997-a9db-5a63afe6f1d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:30.370785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b18e4d-0073-4997-a9db-5a63afe6f1d1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:30.380317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 197/382] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 22:19:30 2022] GET /identity/v3/role_assignments?group.id=a68391f62e9c4c6bab2251c1627ec554&scope.project.id=ff949bd3888c436789a339dd0a2e16fc&role.id=108664305d2546949f7327b65d4cb667 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:30.403247 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113dc66-ee0f-45cf-9d85-91759e3c4716 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:30.444635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a85e40-e631-4745-90b1-b4fe3e805dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:30.445275 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85e40-e631-4745-90b1-b4fe3e805dff None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:30.445553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85e40-e631-4745-90b1-b4fe3e805dff None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:30.445823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85e40-e631-4745-90b1-b4fe3e805dff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:30.461886 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a85e40-e631-4745-90b1-b4fe3e805dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-b8a85e40-e631-4745-90b1-b4fe3e805dff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:30.467387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:30.468314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:30.469811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 186/383] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:30.474474 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b18e4d-0073-4997-a9db-5a63afe6f1d1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:30.563591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:30.598111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b18e4d-0073-4997-a9db-5a63afe6f1d1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:30.639529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32b35-2ebf-48c4-8c3a-b9c793bd8193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:30.640181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32b35-2ebf-48c4-8c3a-b9c793bd8193 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:30.640469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32b35-2ebf-48c4-8c3a-b9c793bd8193 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:30.640738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32b35-2ebf-48c4-8c3a-b9c793bd8193 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:30.641412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca32b35-2ebf-48c4-8c3a-b9c793bd8193 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:30.641714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca32b35-2ebf-48c4-8c3a-b9c793bd8193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:30.643099 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca32b35-2ebf-48c4-8c3a-b9c793bd8193 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:30.653476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 198/384] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:30.659612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a85e40-e631-4745-90b1-b4fe3e805dff None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:30.804699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a85e40-e631-4745-90b1-b4fe3e805dff None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:30.835969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:30.849425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdf7b6-b223-4fce-9e04-b854ddb936a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:30.849991 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdf7b6-b223-4fce-9e04-b854ddb936a7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:30.850272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdf7b6-b223-4fce-9e04-b854ddb936a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:30.850546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdf7b6-b223-4fce-9e04-b854ddb936a7 None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:30.851694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdf7b6-b223-4fce-9e04-b854ddb936a7 None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:30.853173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdf7b6-b223-4fce-9e04-b854ddb936a7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:30.854954 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 187/385] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:30 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:30.860846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32b35-2ebf-48c4-8c3a-b9c793bd8193 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:31.010851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32b35-2ebf-48c4-8c3a-b9c793bd8193 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:31.052590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346a853-bfcb-4aad-af51-c86441930323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:31.053237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346a853-bfcb-4aad-af51-c86441930323 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:31.053518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346a853-bfcb-4aad-af51-c86441930323 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:31.053796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346a853-bfcb-4aad-af51-c86441930323 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:31.054973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346a853-bfcb-4aad-af51-c86441930323 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:31.056442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346a853-bfcb-4aad-af51-c86441930323 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:31.058760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 199/386] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:30 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:31.064668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdf7b6-b223-4fce-9e04-b854ddb936a7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:31.087081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fc051-fbd3-4359-ab6d-f1669ec8e55b None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:31.087397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fc051-fbd3-4359-ab6d-f1669ec8e55b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:31.087824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fc051-fbd3-4359-ab6d-f1669ec8e55b None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:31.089785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 200/387] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:31.095307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b313b-eb49-44a6-8f7b-b81fd6159454 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:31.095593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b313b-eb49-44a6-8f7b-b81fd6159454 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:31.096187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b313b-eb49-44a6-8f7b-b81fd6159454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:31.107206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:31.123003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4b0b313b-eb49-44a6-8f7b-b81fd6159454 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:31.124429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4b0b313b-eb49-44a6-8f7b-b81fd6159454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:31.230567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdf7b6-b223-4fce-9e04-b854ddb936a7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:31.246144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 201/388] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:31.270331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b313b-eb49-44a6-8f7b-b81fd6159454 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:31.287860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd203ae-ddd7-49c7-b0c5-56b6e6de86ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:31.288577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd203ae-ddd7-49c7-b0c5-56b6e6de86ab None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:31.288910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd203ae-ddd7-49c7-b0c5-56b6e6de86ab None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:31.289188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd203ae-ddd7-49c7-b0c5-56b6e6de86ab None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/groups/405ae5a729404d04880e6c6b9ed925c3/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:31.296816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd203ae-ddd7-49c7-b0c5-56b6e6de86ab None admin] RBAC: Authorizing `identity:create_grant(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7, group_id=405ae5a729404d04880e6c6b9ed925c3, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:31.298525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd203ae-ddd7-49c7-b0c5-56b6e6de86ab None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:31.387483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 188/389] 10.222.0.30 () {58 vars in 1442 bytes} [Sun Aug 7 22:19:31 2022] PUT /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/groups/405ae5a729404d04880e6c6b9ed925c3/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:31.401523 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b313b-eb49-44a6-8f7b-b81fd6159454 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:31.437424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:31.447495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce496-0e24-4adf-bd0f-66f121d23113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:31.448925 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce496-0e24-4adf-bd0f-66f121d23113 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:31.449203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce496-0e24-4adf-bd0f-66f121d23113 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:31.450670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce496-0e24-4adf-bd0f-66f121d23113 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:31.463769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ce496-0e24-4adf-bd0f-66f121d23113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-288ce496-0e24-4adf-bd0f-66f121d23113 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:31.469546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:31.470409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:31.471211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 202/390] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:19:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:31.478341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd203ae-ddd7-49c7-b0c5-56b6e6de86ab None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:31.568397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:31.621508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd203ae-ddd7-49c7-b0c5-56b6e6de86ab None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:31.662048 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fc0d5-8f9d-48b1-80b0-8862a5cbcaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:31.662636 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc0d5-8f9d-48b1-80b0-8862a5cbcaaa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:31.662871 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc0d5-8f9d-48b1-80b0-8862a5cbcaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:31.663155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc0d5-8f9d-48b1-80b0-8862a5cbcaaa None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:31.663840 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fc0d5-8f9d-48b1-80b0-8862a5cbcaaa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:31.664177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fc0d5-8f9d-48b1-80b0-8862a5cbcaaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:31.665530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fc0d5-8f9d-48b1-80b0-8862a5cbcaaa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:31.676471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 189/391] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:31.682036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce496-0e24-4adf-bd0f-66f121d23113 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:31.821071 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce496-0e24-4adf-bd0f-66f121d23113 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:31.865629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1875a4-a9c7-4016-96ca-da7d03f985a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:31.866271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1875a4-a9c7-4016-96ca-da7d03f985a5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:31.866550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1875a4-a9c7-4016-96ca-da7d03f985a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:31.866824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1875a4-a9c7-4016-96ca-da7d03f985a5 None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:31.867949 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1875a4-a9c7-4016-96ca-da7d03f985a5 None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:31.869478 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1875a4-a9c7-4016-96ca-da7d03f985a5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:31.871778 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 203/392] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:31 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:31.877091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fc0d5-8f9d-48b1-80b0-8862a5cbcaaa None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:32.004869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24286c5-7c96-4983-a59c-dd8b2ec4a3ac None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:32.005193 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24286c5-7c96-4983-a59c-dd8b2ec4a3ac None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:32.005432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24286c5-7c96-4983-a59c-dd8b2ec4a3ac None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:32.008620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 204/393] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:32.013433 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc6c62-fb0b-4893-b117-96fd686c9616 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:32.013720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc6c62-fb0b-4893-b117-96fd686c9616 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:32.014001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc6c62-fb0b-4893-b117-96fd686c9616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:32.019375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fc0d5-8f9d-48b1-80b0-8862a5cbcaaa None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:32.042141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:32.060046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-eebc6c62-fb0b-4893-b117-96fd686c9616 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:32.060046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-eebc6c62-fb0b-4893-b117-96fd686c9616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:32.067307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2475e3-d9ca-4ec3-b70e-d26f50657a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:32.068283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2475e3-d9ca-4ec3-b70e-d26f50657a93 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:32.068767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2475e3-d9ca-4ec3-b70e-d26f50657a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:32.069766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2475e3-d9ca-4ec3-b70e-d26f50657a93 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:32.074282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2475e3-d9ca-4ec3-b70e-d26f50657a93 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:32.076479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2475e3-d9ca-4ec3-b70e-d26f50657a93 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:32.078692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 190/394] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:31 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:32.084465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2475e3-d9ca-4ec3-b70e-d26f50657a93 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:32.207393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:32.218243 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 205/395] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:32.229357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2475e3-d9ca-4ec3-b70e-d26f50657a93 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:32.262452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc6c62-fb0b-4893-b117-96fd686c9616 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:32.292170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab49296-9676-45fc-95a5-7c2aa190b399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:32.292170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab49296-9676-45fc-95a5-7c2aa190b399 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:32.292170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab49296-9676-45fc-95a5-7c2aa190b399 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:32.292170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab49296-9676-45fc-95a5-7c2aa190b399 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/groups/405ae5a729404d04880e6c6b9ed925c3/roles/b69a8899fce14a0d839745bb76b0f441` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:32.292575 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab49296-9676-45fc-95a5-7c2aa190b399 None admin] RBAC: Authorizing `identity:create_grant(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7, group_id=405ae5a729404d04880e6c6b9ed925c3, role_id=b69a8899fce14a0d839745bb76b0f441)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:32.292575 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab49296-9676-45fc-95a5-7c2aa190b399 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:32.359997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 191/396] 10.222.0.30 () {58 vars in 1442 bytes} [Sun Aug 7 22:19:32 2022] PUT /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7/groups/405ae5a729404d04880e6c6b9ed925c3/roles/b69a8899fce14a0d839745bb76b0f441 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:19:32.407877 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc6c62-fb0b-4893-b117-96fd686c9616 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:32.460644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4751d-9c2d-493e-b27d-138ea01270da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:32.461259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4751d-9c2d-493e-b27d-138ea01270da None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:32.461538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4751d-9c2d-493e-b27d-138ea01270da None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:32.461911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4751d-9c2d-493e-b27d-138ea01270da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:32.478644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4751d-9c2d-493e-b27d-138ea01270da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-82f4751d-9c2d-493e-b27d-138ea01270da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:32.485358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:32.486303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:32.487157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 206/397] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:32.494235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab49296-9676-45fc-95a5-7c2aa190b399 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:32.554177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:32.620947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab49296-9676-45fc-95a5-7c2aa190b399 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:32.666803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc6dca-d784-47db-bb75-b15173273684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:32.667703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc6dca-d784-47db-bb75-b15173273684 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:32.667884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc6dca-d784-47db-bb75-b15173273684 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:32.668057 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc6dca-d784-47db-bb75-b15173273684 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:32.668689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc6dca-d784-47db-bb75-b15173273684 None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:32.668987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc6dca-d784-47db-bb75-b15173273684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:32.670403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc6dca-d784-47db-bb75-b15173273684 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:32.687252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 192/398] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:32.693110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4751d-9c2d-493e-b27d-138ea01270da None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:32.783486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:32.821850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69ca5f-6cef-4c76-954e-8c3339168ed7 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:32.822177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69ca5f-6cef-4c76-954e-8c3339168ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:32.822417 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69ca5f-6cef-4c76-954e-8c3339168ed7 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:32.827850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 193/399] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:32.829219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1afab-dcc3-4e59-a66b-32d040396d36 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:32.829500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1afab-dcc3-4e59-a66b-32d040396d36 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:32.829773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1afab-dcc3-4e59-a66b-32d040396d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:32.858314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-94f1afab-dcc3-4e59-a66b-32d040396d36 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:32.859956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4751d-9c2d-493e-b27d-138ea01270da None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:32.860188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-94f1afab-dcc3-4e59-a66b-32d040396d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:32.899149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5da41-7785-4eab-bf5a-8b9c56cb1096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:32.899788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5da41-7785-4eab-bf5a-8b9c56cb1096 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:32.900325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5da41-7785-4eab-bf5a-8b9c56cb1096 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:32.901287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5da41-7785-4eab-bf5a-8b9c56cb1096 None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:32.902555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5da41-7785-4eab-bf5a-8b9c56cb1096 None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:32.904098 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5da41-7785-4eab-bf5a-8b9c56cb1096 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:32.906203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 207/400] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:32 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:32.911508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5da41-7785-4eab-bf5a-8b9c56cb1096 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:33.031667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 194/401] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:33.062224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1afab-dcc3-4e59-a66b-32d040396d36 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:33.106847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5da41-7785-4eab-bf5a-8b9c56cb1096 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:33.128459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:33.129700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:33.149562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a9a20-3701-4e89-a8a5-43e0fcf9f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:33.151189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a9a20-3701-4e89-a8a5-43e0fcf9f3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:33.151675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a9a20-3701-4e89-a8a5-43e0fcf9f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:33.152302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a9a20-3701-4e89-a8a5-43e0fcf9f3f9 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:33.154140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a9a20-3701-4e89-a8a5-43e0fcf9f3f9 None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:33.155877 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a9a20-3701-4e89-a8a5-43e0fcf9f3f9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:33.158060 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 208/402] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:32 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:33.164879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a9a20-3701-4e89-a8a5-43e0fcf9f3f9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:33.280540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a9a20-3701-4e89-a8a5-43e0fcf9f3f9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:33.317465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239491e1-2f31-4331-9283-099ad60ffac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:33.318627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239491e1-2f31-4331-9283-099ad60ffac7 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:33.319108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239491e1-2f31-4331-9283-099ad60ffac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:33.319579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239491e1-2f31-4331-9283-099ad60ffac7 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/groups/405ae5a729404d04880e6c6b9ed925c3/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:33.322245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239491e1-2f31-4331-9283-099ad60ffac7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3, group_id=405ae5a729404d04880e6c6b9ed925c3, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:33.324910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239491e1-2f31-4331-9283-099ad60ffac7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:33.333952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1afab-dcc3-4e59-a66b-32d040396d36 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:33.372006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb403323-cf24-420d-b944-65e7e684dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:33.372926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb403323-cf24-420d-b944-65e7e684dd45 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:33.373352 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb403323-cf24-420d-b944-65e7e684dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:33.373800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb403323-cf24-420d-b944-65e7e684dd45 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:33.386584 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb403323-cf24-420d-b944-65e7e684dd45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-bb403323-cf24-420d-b944-65e7e684dd45 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:33.391893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:33.393039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:33.394062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:33.396473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 195/403] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:19:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:33.398085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55375b91-c442-4c6b-9cdb-f9e20cf00684 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:33.398667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55375b91-c442-4c6b-9cdb-f9e20cf00684 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:33.399032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55375b91-c442-4c6b-9cdb-f9e20cf00684 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:33.400627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 196/404] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:33.402551 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55375b91-c442-4c6b-9cdb-f9e20cf00684 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:33.426863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 209/405] 10.222.0.30 () {58 vars in 1442 bytes} [Sun Aug 7 22:19:33 2022] PUT /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/groups/405ae5a729404d04880e6c6b9ed925c3/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:33.429283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647fa1d-0ff0-41c8-b58c-2a250c306313 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:33.429681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647fa1d-0ff0-41c8-b58c-2a250c306313 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:33.430063 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647fa1d-0ff0-41c8-b58c-2a250c306313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:33.461304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d647fa1d-0ff0-41c8-b58c-2a250c306313 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:33.462736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d647fa1d-0ff0-41c8-b58c-2a250c306313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:33.528666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:33.568771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 210/406] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:33.592244 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647fa1d-0ff0-41c8-b58c-2a250c306313 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:33.618827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:33.639926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55375b91-c442-4c6b-9cdb-f9e20cf00684 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:33.708535 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6128b1-ed5f-4c4f-af3b-173c9e1258ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:33.709198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6128b1-ed5f-4c4f-af3b-173c9e1258ce None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:33.709277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6128b1-ed5f-4c4f-af3b-173c9e1258ce None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:33.709516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6128b1-ed5f-4c4f-af3b-173c9e1258ce None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:33.710170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6128b1-ed5f-4c4f-af3b-173c9e1258ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:33.710481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6128b1-ed5f-4c4f-af3b-173c9e1258ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:33.711860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6128b1-ed5f-4c4f-af3b-173c9e1258ce None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:33.722874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 197/407] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:33.725585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647fa1d-0ff0-41c8-b58c-2a250c306313 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:33.728261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6128b1-ed5f-4c4f-af3b-173c9e1258ce None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:33.779469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafe751-6434-4f31-bdcf-0dc5f62eb0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:33.780142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafe751-6434-4f31-bdcf-0dc5f62eb0ef None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:33.780420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafe751-6434-4f31-bdcf-0dc5f62eb0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:33.780691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafe751-6434-4f31-bdcf-0dc5f62eb0ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:33.794314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafe751-6434-4f31-bdcf-0dc5f62eb0ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-9fafe751-6434-4f31-bdcf-0dc5f62eb0ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:33.801824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:33.802715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:33.803514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:33.808127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 211/408] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:19:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:33.810249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafe751-6434-4f31-bdcf-0dc5f62eb0ef None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:33.883070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6128b1-ed5f-4c4f-af3b-173c9e1258ce None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:33.889117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:33.923253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065e0fe-9b21-4b58-91ed-f5a72375361e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:33.924311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065e0fe-9b21-4b58-91ed-f5a72375361e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:33.924797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065e0fe-9b21-4b58-91ed-f5a72375361e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:33.925272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065e0fe-9b21-4b58-91ed-f5a72375361e None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:33.926693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065e0fe-9b21-4b58-91ed-f5a72375361e None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:33.928383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065e0fe-9b21-4b58-91ed-f5a72375361e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:33.930658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 198/409] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:33 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:33.939270 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065e0fe-9b21-4b58-91ed-f5a72375361e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:33.988068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:34.109787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafe751-6434-4f31-bdcf-0dc5f62eb0ef None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:34.119422 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065e0fe-9b21-4b58-91ed-f5a72375361e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:34.176999 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9920-7ddb-44e3-a543-f30d850fdfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:34.178696 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9920-7ddb-44e3-a543-f30d850fdfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:34.179547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9920-7ddb-44e3-a543-f30d850fdfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:34.180183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9920-7ddb-44e3-a543-f30d850fdfc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:34.180872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9920-7ddb-44e3-a543-f30d850fdfc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:34.181182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9920-7ddb-44e3-a543-f30d850fdfc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:34.183298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9920-7ddb-44e3-a543-f30d850fdfc2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:34.192337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0674ef-135a-4ef5-a9da-621e387c16c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:34.192526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 212/410] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:34.193597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0674ef-135a-4ef5-a9da-621e387c16c0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:34.194055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0674ef-135a-4ef5-a9da-621e387c16c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:34.194503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0674ef-135a-4ef5-a9da-621e387c16c0 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:34.196784 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0674ef-135a-4ef5-a9da-621e387c16c0 None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:34.198461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71773e88-89e8-46b2-ace3-56c94853d86a None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:34.198747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71773e88-89e8-46b2-ace3-56c94853d86a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:34.199029 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71773e88-89e8-46b2-ace3-56c94853d86a None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:34.199188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0674ef-135a-4ef5-a9da-621e387c16c0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:34.204634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 213/411] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:34.204634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71773e88-89e8-46b2-ace3-56c94853d86a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:34.205317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 199/412] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:33 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:34.206585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54646e2f-543c-4029-b5f6-18accffe3c01 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:34.206869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54646e2f-543c-4029-b5f6-18accffe3c01 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:34.207143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54646e2f-543c-4029-b5f6-18accffe3c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:34.226598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-54646e2f-543c-4029-b5f6-18accffe3c01 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:34.227784 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-54646e2f-543c-4029-b5f6-18accffe3c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:34.302901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 200/413] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:34.304078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54646e2f-543c-4029-b5f6-18accffe3c01 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:34.316959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:34.321805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71773e88-89e8-46b2-ace3-56c94853d86a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:34.333066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:34.356220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243819f-5c1a-4c31-a34c-0a382a93820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:34.356846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243819f-5c1a-4c31-a34c-0a382a93820b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:34.357122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243819f-5c1a-4c31-a34c-0a382a93820b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:34.357393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243819f-5c1a-4c31-a34c-0a382a93820b None admin] PATH_INFO: `/v3/groups/a68391f62e9c4c6bab2251c1627ec554` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:34.358552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243819f-5c1a-4c31-a34c-0a382a93820b None admin] RBAC: Authorizing `identity:get_group(group_id=a68391f62e9c4c6bab2251c1627ec554)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:34.360241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243819f-5c1a-4c31-a34c-0a382a93820b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:34.362077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 214/414] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:34 2022] GET /identity/v3/groups/a68391f62e9c4c6bab2251c1627ec554 => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:34.362851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243819f-5c1a-4c31-a34c-0a382a93820b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:34.412040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54646e2f-543c-4029-b5f6-18accffe3c01 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:34.447904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe115a7-762c-42de-8faf-52a15a62b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:34.448932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe115a7-762c-42de-8faf-52a15a62b8eb None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:34.449388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe115a7-762c-42de-8faf-52a15a62b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:34.449834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe115a7-762c-42de-8faf-52a15a62b8eb None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/groups/405ae5a729404d04880e6c6b9ed925c3/roles/b69a8899fce14a0d839745bb76b0f441` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:34.452231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe115a7-762c-42de-8faf-52a15a62b8eb None admin] RBAC: Authorizing `identity:create_grant(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3, group_id=405ae5a729404d04880e6c6b9ed925c3, role_id=b69a8899fce14a0d839745bb76b0f441)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:34.453858 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe115a7-762c-42de-8faf-52a15a62b8eb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:34.469783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243819f-5c1a-4c31-a34c-0a382a93820b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:34.505835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43e13f-3097-4b3f-85fe-f3654ae23fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:34.506460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43e13f-3097-4b3f-85fe-f3654ae23fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:34.506752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43e13f-3097-4b3f-85fe-f3654ae23fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:34.507089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43e13f-3097-4b3f-85fe-f3654ae23fa2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:34.516818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 201/415] 10.222.0.30 () {58 vars in 1442 bytes} [Sun Aug 7 22:19:34 2022] PUT /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3/groups/405ae5a729404d04880e6c6b9ed925c3/roles/b69a8899fce14a0d839745bb76b0f441 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:34.518126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe115a7-762c-42de-8faf-52a15a62b8eb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:34.528762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d43e13f-3097-4b3f-85fe-f3654ae23fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-2d43e13f-3097-4b3f-85fe-f3654ae23fa2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:34.533769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:34.534680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:34.535992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 215/416] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:34.541334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43e13f-3097-4b3f-85fe-f3654ae23fa2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:34.602031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:34.640211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:34.658816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe115a7-762c-42de-8faf-52a15a62b8eb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:34.701429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e1c3d-ca2e-40a6-b2f5-71a9159efc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:34.702346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e1c3d-ca2e-40a6-b2f5-71a9159efc3e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:34.702747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e1c3d-ca2e-40a6-b2f5-71a9159efc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:34.703354 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e1c3d-ca2e-40a6-b2f5-71a9159efc3e None admin] PATH_INFO: `/v3/projects/ff949bd3888c436789a339dd0a2e16fc` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:34.704706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e1c3d-ca2e-40a6-b2f5-71a9159efc3e None admin] RBAC: Authorizing `identity:get_project(project_id=ff949bd3888c436789a339dd0a2e16fc)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:34.706467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e1c3d-ca2e-40a6-b2f5-71a9159efc3e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:34.709432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 202/417] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:34 2022] GET /identity/v3/projects/ff949bd3888c436789a339dd0a2e16fc => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:34.716816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e1c3d-ca2e-40a6-b2f5-71a9159efc3e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:34.778766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43e13f-3097-4b3f-85fe-f3654ae23fa2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:34.821015 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f1d4b-557b-45aa-b337-940a224546a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:34.821624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f1d4b-557b-45aa-b337-940a224546a6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:34.821910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f1d4b-557b-45aa-b337-940a224546a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:34.822182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f1d4b-557b-45aa-b337-940a224546a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:34.823932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f1d4b-557b-45aa-b337-940a224546a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:34.824269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f1d4b-557b-45aa-b337-940a224546a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:34.825662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f1d4b-557b-45aa-b337-940a224546a6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:34.840117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 216/418] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:34.841996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f1d4b-557b-45aa-b337-940a224546a6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:34.852980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e1c3d-ca2e-40a6-b2f5-71a9159efc3e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:34.895977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea5558-532a-4ec1-9f64-b87e69c13be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:34.896682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea5558-532a-4ec1-9f64-b87e69c13be3 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:34.896965 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea5558-532a-4ec1-9f64-b87e69c13be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:34.897243 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea5558-532a-4ec1-9f64-b87e69c13be3 None admin] PATH_INFO: `/v3/projects/ff949bd3888c436789a339dd0a2e16fc/groups/a68391f62e9c4c6bab2251c1627ec554/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:34.899661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea5558-532a-4ec1-9f64-b87e69c13be3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff949bd3888c436789a339dd0a2e16fc, group_id=a68391f62e9c4c6bab2251c1627ec554, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:34.901704 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea5558-532a-4ec1-9f64-b87e69c13be3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:34.965240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:34.982380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 203/419] 10.222.0.30 () {58 vars in 1442 bytes} [Sun Aug 7 22:19:34 2022] PUT /identity/v3/projects/ff949bd3888c436789a339dd0a2e16fc/groups/a68391f62e9c4c6bab2251c1627ec554/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:34.987078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f1d4b-557b-45aa-b337-940a224546a6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:35.027439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1550e46-f037-46a6-b4ce-ff46e6f9ac2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:35.028108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1550e46-f037-46a6-b4ce-ff46e6f9ac2e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:35.028392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1550e46-f037-46a6-b4ce-ff46e6f9ac2e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:35.028670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1550e46-f037-46a6-b4ce-ff46e6f9ac2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:35.052243 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bf992-fa13-4ca2-adcf-995b0ead6bb9 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:35.052564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bf992-fa13-4ca2-adcf-995b0ead6bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:35.052798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bf992-fa13-4ca2-adcf-995b0ead6bb9 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:35.054391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 204/420] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:35.061926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505869f-431c-45eb-a120-090e3db948ad None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:35.062204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505869f-431c-45eb-a120-090e3db948ad None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:35.062477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505869f-431c-45eb-a120-090e3db948ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:35.097272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c505869f-431c-45eb-a120-090e3db948ad None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:35.098496 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c505869f-431c-45eb-a120-090e3db948ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:35.113353 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1550e46-f037-46a6-b4ce-ff46e6f9ac2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:35.115328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1550e46-f037-46a6-b4ce-ff46e6f9ac2e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:35.124391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:35.233425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 205/421] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:35.245294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:35.259427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 217/422] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:35.260427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505869f-431c-45eb-a120-090e3db948ad None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:35.265161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1550e46-f037-46a6-b4ce-ff46e6f9ac2e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:35.371960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505869f-431c-45eb-a120-090e3db948ad None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:35.393637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1550e46-f037-46a6-b4ce-ff46e6f9ac2e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:35.408595 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207424ef-f934-4800-a5ca-e4042c484abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:35.409650 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207424ef-f934-4800-a5ca-e4042c484abf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:35.410185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207424ef-f934-4800-a5ca-e4042c484abf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:35.410649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207424ef-f934-4800-a5ca-e4042c484abf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:35.421991 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207424ef-f934-4800-a5ca-e4042c484abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-207424ef-f934-4800-a5ca-e4042c484abf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:35.427351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:35.428285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:35.429109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:35.431746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 206/423] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:19:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:35.437915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207424ef-f934-4800-a5ca-e4042c484abf None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:35.455400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:35.467389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cf266-43ed-458e-878d-c950d5e464a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:35.473178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cf266-43ed-458e-878d-c950d5e464a5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:35.473178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cf266-43ed-458e-878d-c950d5e464a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:35.473178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cf266-43ed-458e-878d-c950d5e464a5 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:35.473178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8cf266-43ed-458e-878d-c950d5e464a5 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:35.473178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8cf266-43ed-458e-878d-c950d5e464a5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:35.477557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 218/424] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:35 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:35.481418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cf266-43ed-458e-878d-c950d5e464a5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:35.560388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207424ef-f934-4800-a5ca-e4042c484abf None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:35.596978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f55b4c-1766-4344-a62a-114fc299e5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:35.598212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55b4c-1766-4344-a62a-114fc299e5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:35.598692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55b4c-1766-4344-a62a-114fc299e5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:35.599167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55b4c-1766-4344-a62a-114fc299e5f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:35.600048 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f55b4c-1766-4344-a62a-114fc299e5f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:35.600616 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f55b4c-1766-4344-a62a-114fc299e5f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:35.603071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f55b4c-1766-4344-a62a-114fc299e5f4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:35.613022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 207/425] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:35.620902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f55b4c-1766-4344-a62a-114fc299e5f4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:35.673326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:35.696370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:35.733869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f55b4c-1766-4344-a62a-114fc299e5f4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:35.772485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8da6f-5701-46b3-9784-44aef6da6dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:35.773495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8da6f-5701-46b3-9784-44aef6da6dfa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:35.774002 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8da6f-5701-46b3-9784-44aef6da6dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:35.774489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8da6f-5701-46b3-9784-44aef6da6dfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:35.811351 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cf266-43ed-458e-878d-c950d5e464a5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:35.871902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b6675-cdae-40f2-abef-57852cb6e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:35.872558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b6675-cdae-40f2-abef-57852cb6e359 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:35.872837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b6675-cdae-40f2-abef-57852cb6e359 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:35.873111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b6675-cdae-40f2-abef-57852cb6e359 None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:35.877623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b6675-cdae-40f2-abef-57852cb6e359 None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:35.880321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b6675-cdae-40f2-abef-57852cb6e359 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:35.882469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8da6f-5701-46b3-9784-44aef6da6dfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:35.889553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 219/426] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:35 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:35.889777 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8da6f-5701-46b3-9784-44aef6da6dfa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:35.892934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b6675-cdae-40f2-abef-57852cb6e359 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:36.003758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:36.042937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 208/427] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 22:19:36.048693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd756fd7-dac6-4096-99e3-d88a089f1f24 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:36.049202 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd756fd7-dac6-4096-99e3-d88a089f1f24 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:36.049688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd756fd7-dac6-4096-99e3-d88a089f1f24 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:36.051209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 209/428] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:36.052721 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd756fd7-dac6-4096-99e3-d88a089f1f24 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:36.065012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b6675-cdae-40f2-abef-57852cb6e359 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:36.097384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:36.112477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d6678-221b-4d38-9e84-f3fc55487d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:36.113149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d6678-221b-4d38-9e84-f3fc55487d6c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:36.113402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d6678-221b-4d38-9e84-f3fc55487d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:36.113681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d6678-221b-4d38-9e84-f3fc55487d6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:36.114406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d6678-221b-4d38-9e84-f3fc55487d6c None admin] RBAC: Adding query filter params (group.id=405ae5a729404d04880e6c6b9ed925c3, role.id=87ab4761555e43419d0f23a0f31114f3, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:36.114969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d6678-221b-4d38-9e84-f3fc55487d6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:36.117552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d6678-221b-4d38-9e84-f3fc55487d6c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:36.127936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 220/429] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 22:19:35 2022] GET /identity/v3/role_assignments?group.id=405ae5a729404d04880e6c6b9ed925c3&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 611 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:36.128984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b0a57-f2dc-4342-9007-fdae0caa0be2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:36.129263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b0a57-f2dc-4342-9007-fdae0caa0be2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:36.129531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b0a57-f2dc-4342-9007-fdae0caa0be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:36.173039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-436b0a57-f2dc-4342-9007-fdae0caa0be2 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:36.178998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-436b0a57-f2dc-4342-9007-fdae0caa0be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:36.228690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd756fd7-dac6-4096-99e3-d88a089f1f24 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:36.284763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b980c2-3337-4444-a951-f8f214b90b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:36.285428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b980c2-3337-4444-a951-f8f214b90b25 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:36.286152 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b980c2-3337-4444-a951-f8f214b90b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:36.286478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b980c2-3337-4444-a951-f8f214b90b25 None admin] PATH_INFO: `/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:36.287889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b980c2-3337-4444-a951-f8f214b90b25 None admin] RBAC: Authorizing `identity:get_project(project_id=02a9a45b37ec4042a0e02d0ae5b3fdb7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:36.290003 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b980c2-3337-4444-a951-f8f214b90b25 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:36.291896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 210/430] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:36 2022] GET /identity/v3/projects/02a9a45b37ec4042a0e02d0ae5b3fdb7 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:36.298081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b980c2-3337-4444-a951-f8f214b90b25 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:36.316825 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 221/431] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:36.337253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:36.349442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b0a57-f2dc-4342-9007-fdae0caa0be2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:36.406539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b980c2-3337-4444-a951-f8f214b90b25 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:36.442817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27934a15-22ed-4407-8c8c-b74b039c02dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:36.443887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27934a15-22ed-4407-8c8c-b74b039c02dd None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:36.443887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27934a15-22ed-4407-8c8c-b74b039c02dd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:36.444085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27934a15-22ed-4407-8c8c-b74b039c02dd None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:36.446236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27934a15-22ed-4407-8c8c-b74b039c02dd None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:36.449290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27934a15-22ed-4407-8c8c-b74b039c02dd None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:36.455675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 211/432] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:36 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:36.457110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27934a15-22ed-4407-8c8c-b74b039c02dd None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:36.476053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b0a57-f2dc-4342-9007-fdae0caa0be2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:36.504370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:36.518983 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba21fe-2953-457c-9e4d-6807d5550366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:36.519719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba21fe-2953-457c-9e4d-6807d5550366 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:36.520005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba21fe-2953-457c-9e4d-6807d5550366 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:36.520431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba21fe-2953-457c-9e4d-6807d5550366 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:36.532733 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba21fe-2953-457c-9e4d-6807d5550366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-55ba21fe-2953-457c-9e4d-6807d5550366 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:36.538766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:36.539662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:36.540512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 222/433] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:36.546738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba21fe-2953-457c-9e4d-6807d5550366 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:36.563101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27934a15-22ed-4407-8c8c-b74b039c02dd None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:36.566757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:36.608770 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef24032-86fd-474b-9ac6-d0d30cbe37a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:36.609403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef24032-86fd-474b-9ac6-d0d30cbe37a7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:36.609745 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef24032-86fd-474b-9ac6-d0d30cbe37a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:36.610085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef24032-86fd-474b-9ac6-d0d30cbe37a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:36.610979 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef24032-86fd-474b-9ac6-d0d30cbe37a7 None admin] RBAC: Adding query filter params (group.id=405ae5a729404d04880e6c6b9ed925c3, role.id=b69a8899fce14a0d839745bb76b0f441, scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:36.611835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef24032-86fd-474b-9ac6-d0d30cbe37a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:36.613673 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef24032-86fd-474b-9ac6-d0d30cbe37a7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:36.624131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 212/434] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 22:19:36 2022] GET /identity/v3/role_assignments?group.id=405ae5a729404d04880e6c6b9ed925c3&scope.project.id=02a9a45b37ec4042a0e02d0ae5b3fdb7&role.id=b69a8899fce14a0d839745bb76b0f441 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:36.671950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba21fe-2953-457c-9e4d-6807d5550366 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:36.712449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09afc05-f289-43b2-bbe6-8c14484596a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:36.714165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09afc05-f289-43b2-bbe6-8c14484596a5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:36.714624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09afc05-f289-43b2-bbe6-8c14484596a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:36.715092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09afc05-f289-43b2-bbe6-8c14484596a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:36.715962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09afc05-f289-43b2-bbe6-8c14484596a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:36.716500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09afc05-f289-43b2-bbe6-8c14484596a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:36.718188 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09afc05-f289-43b2-bbe6-8c14484596a5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:36.731936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 223/435] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:19:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:36.734488 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef24032-86fd-474b-9ac6-d0d30cbe37a7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:36.847200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef24032-86fd-474b-9ac6-d0d30cbe37a7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:36.882075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cae5bb-260b-4c6d-9618-e5176069f854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:36.882967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cae5bb-260b-4c6d-9618-e5176069f854 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:36.883418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cae5bb-260b-4c6d-9618-e5176069f854 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:36.883872 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cae5bb-260b-4c6d-9618-e5176069f854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:36.920752 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:36.939972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225479-ece0-4355-9313-69430af104de None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:36.940558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225479-ece0-4355-9313-69430af104de None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:36.940558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225479-ece0-4355-9313-69430af104de None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:36.942189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 224/436] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:36.950583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f7cd-4fdd-4f2d-8d6e-f12855eac9be None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:36.950583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f7cd-4fdd-4f2d-8d6e-f12855eac9be None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:36.950583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f7cd-4fdd-4f2d-8d6e-f12855eac9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:36.973478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cae5bb-260b-4c6d-9618-e5176069f854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:36.975390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cae5bb-260b-4c6d-9618-e5176069f854 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:36.986387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2ff6f7cd-4fdd-4f2d-8d6e-f12855eac9be None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:36.988509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2ff6f7cd-4fdd-4f2d-8d6e-f12855eac9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:37.039999 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:37.083770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 225/437] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:37.108257 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6f7cd-4fdd-4f2d-8d6e-f12855eac9be None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:37.111762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 213/438] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:37.117028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cae5bb-260b-4c6d-9618-e5176069f854 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:37.197878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:37.240887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cae5bb-260b-4c6d-9618-e5176069f854 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:37.255626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6f7cd-4fdd-4f2d-8d6e-f12855eac9be None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:37.282061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdedec-5ed8-485b-bc20-819ed0625777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:37.282847 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdedec-5ed8-485b-bc20-819ed0625777 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:37.283131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdedec-5ed8-485b-bc20-819ed0625777 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:37.283407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdedec-5ed8-485b-bc20-819ed0625777 None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:37.284675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdedec-5ed8-485b-bc20-819ed0625777 None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:37.286107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdedec-5ed8-485b-bc20-819ed0625777 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:37.292824 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 214/439] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:37 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:37.293829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdedec-5ed8-485b-bc20-819ed0625777 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:37.296543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1955a50-467b-4e14-8e8c-4c62ff1d5e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:37.298078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1955a50-467b-4e14-8e8c-4c62ff1d5e4d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:37.298366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1955a50-467b-4e14-8e8c-4c62ff1d5e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:37.298638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1955a50-467b-4e14-8e8c-4c62ff1d5e4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:37.310096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1955a50-467b-4e14-8e8c-4c62ff1d5e4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f1955a50-467b-4e14-8e8c-4c62ff1d5e4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:37.315116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:37.315977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:37.321731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 226/440] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:19:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:37.322427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1955a50-467b-4e14-8e8c-4c62ff1d5e4d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:37.356293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:37.410440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdedec-5ed8-485b-bc20-819ed0625777 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:37.433066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1955a50-467b-4e14-8e8c-4c62ff1d5e4d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:37.445063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d6eed-aaac-462e-91ea-9512481397df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:37.445686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d6eed-aaac-462e-91ea-9512481397df None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:37.445961 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d6eed-aaac-462e-91ea-9512481397df None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:37.446232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d6eed-aaac-462e-91ea-9512481397df None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:37.447402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d6eed-aaac-462e-91ea-9512481397df None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:37.448934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d6eed-aaac-462e-91ea-9512481397df None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:37.450437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 215/441] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:37 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:37.456004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d6eed-aaac-462e-91ea-9512481397df None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:37.458637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:37.471701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb964749-9e2c-4e31-a78b-6c3a925ee0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:37.476604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb964749-9e2c-4e31-a78b-6c3a925ee0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:37.476604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb964749-9e2c-4e31-a78b-6c3a925ee0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:37.476604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb964749-9e2c-4e31-a78b-6c3a925ee0b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:37.476604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb964749-9e2c-4e31-a78b-6c3a925ee0b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:37.476604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb964749-9e2c-4e31-a78b-6c3a925ee0b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:37.476604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb964749-9e2c-4e31-a78b-6c3a925ee0b0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:37.487076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 227/442] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:37.492364 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb964749-9e2c-4e31-a78b-6c3a925ee0b0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:37.560911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d6eed-aaac-462e-91ea-9512481397df None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:37.587053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:37.595442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64340195-0364-4025-afb8-daae65b72bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:37.602329 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb964749-9e2c-4e31-a78b-6c3a925ee0b0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:37.602436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64340195-0364-4025-afb8-daae65b72bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:37.602436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64340195-0364-4025-afb8-daae65b72bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:37.602436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64340195-0364-4025-afb8-daae65b72bf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:37.602436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64340195-0364-4025-afb8-daae65b72bf8 None admin] RBAC: Adding query filter params (group.id=405ae5a729404d04880e6c6b9ed925c3, role.id=87ab4761555e43419d0f23a0f31114f3, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:37.602436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64340195-0364-4025-afb8-daae65b72bf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:37.602436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64340195-0364-4025-afb8-daae65b72bf8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:37.605493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 216/443] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 22:19:37 2022] GET /identity/v3/role_assignments?group.id=405ae5a729404d04880e6c6b9ed925c3&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=87ab4761555e43419d0f23a0f31114f3 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:37.606647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beded907-2093-4835-a6e0-72934aadd4f2 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:37.606938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beded907-2093-4835-a6e0-72934aadd4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:37.607218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beded907-2093-4835-a6e0-72934aadd4f2 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:37.608619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 217/444] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:37.614246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48232d6a-d6bb-43ce-b3b5-caa9e2dfb302 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:37.614576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48232d6a-d6bb-43ce-b3b5-caa9e2dfb302 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:37.614798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48232d6a-d6bb-43ce-b3b5-caa9e2dfb302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:37.633921 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-48232d6a-d6bb-43ce-b3b5-caa9e2dfb302 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:37.635203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-48232d6a-d6bb-43ce-b3b5-caa9e2dfb302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:37.642599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8eeb3f-7a47-40ab-b486-135d09e7754c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:37.642599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8eeb3f-7a47-40ab-b486-135d09e7754c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:37.642599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8eeb3f-7a47-40ab-b486-135d09e7754c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:37.642599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8eeb3f-7a47-40ab-b486-135d09e7754c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:37.708129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:37.716329 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 218/445] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:37.731927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8eeb3f-7a47-40ab-b486-135d09e7754c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:37.733544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8eeb3f-7a47-40ab-b486-135d09e7754c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:37.740524 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48232d6a-d6bb-43ce-b3b5-caa9e2dfb302 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:37.850551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:37.858953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48232d6a-d6bb-43ce-b3b5-caa9e2dfb302 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:37.860823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 228/446] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 22:19:37.866523 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8eeb3f-7a47-40ab-b486-135d09e7754c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:37.901255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a091a8-15d4-4b50-9127-9f4a42c188cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:37.901899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a091a8-15d4-4b50-9127-9f4a42c188cf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:37.902176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a091a8-15d4-4b50-9127-9f4a42c188cf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:37.902448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a091a8-15d4-4b50-9127-9f4a42c188cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:37.919258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a091a8-15d4-4b50-9127-9f4a42c188cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-18a091a8-15d4-4b50-9127-9f4a42c188cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:37.924259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:37.925129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:37.925910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 219/447] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:19:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:37.931471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a091a8-15d4-4b50-9127-9f4a42c188cf None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:37.986197 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8eeb3f-7a47-40ab-b486-135d09e7754c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:37.994503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:38.020850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e3f8b-bced-46ab-acab-fabbd56a479e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:38.021475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e3f8b-bced-46ab-acab-fabbd56a479e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:38.021750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e3f8b-bced-46ab-acab-fabbd56a479e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:38.022023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e3f8b-bced-46ab-acab-fabbd56a479e None admin] PATH_INFO: `/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:38.023154 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e3f8b-bced-46ab-acab-fabbd56a479e None admin] RBAC: Authorizing `identity:get_project(project_id=ab7b2747f3f4498eb4b1e9de5545c9d3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:38.024630 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e3f8b-bced-46ab-acab-fabbd56a479e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:38.027040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 229/448] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:37 2022] GET /identity/v3/projects/ab7b2747f3f4498eb4b1e9de5545c9d3 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:38.032454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e3f8b-bced-46ab-acab-fabbd56a479e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:38.038993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a091a8-15d4-4b50-9127-9f4a42c188cf None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:38.072105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e659a77-8188-4a25-9f4d-a328d867c4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:38.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e659a77-8188-4a25-9f4d-a328d867c4bf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:38.073096 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e659a77-8188-4a25-9f4d-a328d867c4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:38.073361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e659a77-8188-4a25-9f4d-a328d867c4bf None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:38.074014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e659a77-8188-4a25-9f4d-a328d867c4bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:38.074316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e659a77-8188-4a25-9f4d-a328d867c4bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:38.075659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e659a77-8188-4a25-9f4d-a328d867c4bf None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:38.083363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 220/449] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:38.089699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e659a77-8188-4a25-9f4d-a328d867c4bf None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:38.090888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:38.130816 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e3f8b-bced-46ab-acab-fabbd56a479e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:38.163194 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60125b5-5ff5-4ba1-a901-42863c323565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:38.163805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60125b5-5ff5-4ba1-a901-42863c323565 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:38.164105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60125b5-5ff5-4ba1-a901-42863c323565 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:38.164379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60125b5-5ff5-4ba1-a901-42863c323565 None admin] PATH_INFO: `/v3/groups/405ae5a729404d04880e6c6b9ed925c3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:38.165472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60125b5-5ff5-4ba1-a901-42863c323565 None admin] RBAC: Authorizing `identity:get_group(group_id=405ae5a729404d04880e6c6b9ed925c3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:38.167218 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60125b5-5ff5-4ba1-a901-42863c323565 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:38.168795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 230/450] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:38 2022] GET /identity/v3/groups/405ae5a729404d04880e6c6b9ed925c3 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:38.174567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60125b5-5ff5-4ba1-a901-42863c323565 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:38.190079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e659a77-8188-4a25-9f4d-a328d867c4bf None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:38.207617 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:38.224241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0b439-6f61-4ed8-8f94-94ffd70f2762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:38.224846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0b439-6f61-4ed8-8f94-94ffd70f2762 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:38.225126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0b439-6f61-4ed8-8f94-94ffd70f2762 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:38.225399 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0b439-6f61-4ed8-8f94-94ffd70f2762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:38.272859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60125b5-5ff5-4ba1-a901-42863c323565 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:38.292622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0b439-6f61-4ed8-8f94-94ffd70f2762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:38.294072 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0b439-6f61-4ed8-8f94-94ffd70f2762 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:38.298280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:38.306429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fff2ed-9740-4fb0-8671-30311fb3bfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:38.307545 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fff2ed-9740-4fb0-8671-30311fb3bfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:38.307821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fff2ed-9740-4fb0-8671-30311fb3bfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:38.308121 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fff2ed-9740-4fb0-8671-30311fb3bfa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:38.308849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fff2ed-9740-4fb0-8671-30311fb3bfa3 None admin] RBAC: Adding query filter params (group.id=405ae5a729404d04880e6c6b9ed925c3, role.id=b69a8899fce14a0d839745bb76b0f441, scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:38.309154 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fff2ed-9740-4fb0-8671-30311fb3bfa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:38.310566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fff2ed-9740-4fb0-8671-30311fb3bfa3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:38.316121 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 231/451] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 22:19:38 2022] GET /identity/v3/role_assignments?group.id=405ae5a729404d04880e6c6b9ed925c3&scope.project.id=ab7b2747f3f4498eb4b1e9de5545c9d3&role.id=b69a8899fce14a0d839745bb76b0f441 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:38.400317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 221/452] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 22:19:38.406567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fff2ed-9740-4fb0-8671-30311fb3bfa3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:38.509681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fff2ed-9740-4fb0-8671-30311fb3bfa3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:38.543890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc5baf-f918-421a-8a04-70de72c85c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:38.544479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5baf-f918-421a-8a04-70de72c85c1d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:38.544767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5baf-f918-421a-8a04-70de72c85c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:38.545036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5baf-f918-421a-8a04-70de72c85c1d None admin] PATH_INFO: `/v3/projects/ff949bd3888c436789a339dd0a2e16fc` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:38.546236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc5baf-f918-421a-8a04-70de72c85c1d None admin] RBAC: Authorizing `identity:get_project(project_id=ff949bd3888c436789a339dd0a2e16fc)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:38.547784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc5baf-f918-421a-8a04-70de72c85c1d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:38.549441 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 232/453] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:19:38 2022] GET /identity/v3/projects/ff949bd3888c436789a339dd0a2e16fc => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:38.555016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0b439-6f61-4ed8-8f94-94ffd70f2762 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:38.574276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:38.660181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0b439-6f61-4ed8-8f94-94ffd70f2762 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:38.694266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607b475-c1f6-48a0-b038-861e57f6a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:38.694885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607b475-c1f6-48a0-b038-861e57f6a68b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:38.695162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607b475-c1f6-48a0-b038-861e57f6a68b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:38.695435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607b475-c1f6-48a0-b038-861e57f6a68b None admin] PATH_INFO: `/v3/groups/a68391f62e9c4c6bab2251c1627ec554` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:38.696559 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607b475-c1f6-48a0-b038-861e57f6a68b None admin] RBAC: Authorizing `identity:get_group(group_id=a68391f62e9c4c6bab2251c1627ec554)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:38.698002 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607b475-c1f6-48a0-b038-861e57f6a68b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:38.699552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 222/454] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 22:19:38 2022] GET /identity/v3/groups/a68391f62e9c4c6bab2251c1627ec554 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:38.705386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc5baf-f918-421a-8a04-70de72c85c1d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:38.763738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:38.803359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc5baf-f918-421a-8a04-70de72c85c1d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:38.835499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a947b8d-fe99-4009-b082-fcfd9a9c8f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:38.836120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a947b8d-fe99-4009-b082-fcfd9a9c8f1c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:38.836396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a947b8d-fe99-4009-b082-fcfd9a9c8f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:38.836663 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a947b8d-fe99-4009-b082-fcfd9a9c8f1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:38.837373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a947b8d-fe99-4009-b082-fcfd9a9c8f1c None admin] RBAC: Adding query filter params (group.id=a68391f62e9c4c6bab2251c1627ec554, role.id=108664305d2546949f7327b65d4cb667, scope.project.id=ff949bd3888c436789a339dd0a2e16fc) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:38.837676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a947b8d-fe99-4009-b082-fcfd9a9c8f1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:38.839634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a947b8d-fe99-4009-b082-fcfd9a9c8f1c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:38.845137 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 233/455] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 22:19:38 2022] GET /identity/v3/role_assignments?group.id=a68391f62e9c4c6bab2251c1627ec554&scope.project.id=ff949bd3888c436789a339dd0a2e16fc&role.id=108664305d2546949f7327b65d4cb667 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:41.176152 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce35537-0575-4337-89f8-a7c6cf2993cb None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:41.176660 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce35537-0575-4337-89f8-a7c6cf2993cb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:41.176761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce35537-0575-4337-89f8-a7c6cf2993cb None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:41.178323 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 223/456] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:41.183112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1b98-3d7b-44d8-8cfc-361dfc9f548e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:41.183893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1b98-3d7b-44d8-8cfc-361dfc9f548e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:41.184411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1b98-3d7b-44d8-8cfc-361dfc9f548e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:41.211300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-eb5d1b98-3d7b-44d8-8cfc-361dfc9f548e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:41.212900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-eb5d1b98-3d7b-44d8-8cfc-361dfc9f548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:41.304116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 234/457] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:41.327930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce35537-0575-4337-89f8-a7c6cf2993cb None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:41.447428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:41.471051 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce35537-0575-4337-89f8-a7c6cf2993cb None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:41.479274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f655d-3da3-4ce4-aa27-792d12354bf1 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:41.480259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f655d-3da3-4ce4-aa27-792d12354bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:41.480665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f655d-3da3-4ce4-aa27-792d12354bf1 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:41.482150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 235/458] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:41.487519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d5560-3ca5-4370-8f95-ae34a29b99c3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:41.487802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d5560-3ca5-4370-8f95-ae34a29b99c3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:41.488096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d5560-3ca5-4370-8f95-ae34a29b99c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:41.518264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-033d5560-3ca5-4370-8f95-ae34a29b99c3 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:41.519587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-033d5560-3ca5-4370-8f95-ae34a29b99c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:41.549680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338ec1a-c883-45e3-8c02-fb4f8586bcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:41.550313 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338ec1a-c883-45e3-8c02-fb4f8586bcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:41.550590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338ec1a-c883-45e3-8c02-fb4f8586bcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:41.550860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338ec1a-c883-45e3-8c02-fb4f8586bcb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:41.567730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338ec1a-c883-45e3-8c02-fb4f8586bcb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:41.569495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338ec1a-c883-45e3-8c02-fb4f8586bcb7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-7338ec1a-c883-45e3-8c02-fb4f8586bcb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:41.595590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:41.596785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:41.597624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:41.597624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:41.597624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:41.600950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 224/459] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:41.607696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338ec1a-c883-45e3-8c02-fb4f8586bcb7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:41.617839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:41.639483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 236/460] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:41.677615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d5560-3ca5-4370-8f95-ae34a29b99c3 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:41.715477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338ec1a-c883-45e3-8c02-fb4f8586bcb7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:41.756832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507219a-13b1-46fa-b2f7-c6818d939d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:41.757360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507219a-13b1-46fa-b2f7-c6818d939d18 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:41.757637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507219a-13b1-46fa-b2f7-c6818d939d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:41.757932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507219a-13b1-46fa-b2f7-c6818d939d18 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:41.758587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507219a-13b1-46fa-b2f7-c6818d939d18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:41.758892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507219a-13b1-46fa-b2f7-c6818d939d18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:41.760288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507219a-13b1-46fa-b2f7-c6818d939d18 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:41.769435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:41.774887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 225/461] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:41.776243 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95efc8-3f6e-4be7-83bc-61b7f4e8bad6 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:41.776631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95efc8-3f6e-4be7-83bc-61b7f4e8bad6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:41.784843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95efc8-3f6e-4be7-83bc-61b7f4e8bad6 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:41.784843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 226/462] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:41.784843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95efc8-3f6e-4be7-83bc-61b7f4e8bad6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:41.827278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d5560-3ca5-4370-8f95-ae34a29b99c3 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:41.866057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ba00d-ece4-4061-9f20-186aefc53bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:41.866745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ba00d-ece4-4061-9f20-186aefc53bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:41.867168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ba00d-ece4-4061-9f20-186aefc53bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:41.867451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ba00d-ece4-4061-9f20-186aefc53bf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:41.878874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ba00d-ece4-4061-9f20-186aefc53bf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:41.880369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ba00d-ece4-4061-9f20-186aefc53bf8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:41.893293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95efc8-3f6e-4be7-83bc-61b7f4e8bad6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-059ba00d-ece4-4061-9f20-186aefc53bf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:41.896916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:41.897797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:41.898740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:41.898740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:41.898740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:41.899304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 237/463] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:41.900871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a025f06-648b-4617-acc9-46f48a17db68 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:41.900871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a025f06-648b-4617-acc9-46f48a17db68 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:41.902624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a025f06-648b-4617-acc9-46f48a17db68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:41.919762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4a025f06-648b-4617-acc9-46f48a17db68 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:41.920985 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4a025f06-648b-4617-acc9-46f48a17db68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:41.930930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd2978-b8c0-416d-a0a6-043b54a3bcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:41.931538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd2978-b8c0-416d-a0a6-043b54a3bcf1 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:41.931813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd2978-b8c0-416d-a0a6-043b54a3bcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:41.932211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd2978-b8c0-416d-a0a6-043b54a3bcf1 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:41.933381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd2978-b8c0-416d-a0a6-043b54a3bcf1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:41.934997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd2978-b8c0-416d-a0a6-043b54a3bcf1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:41.953631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:41.994138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 238/464] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:42.000912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a025f06-648b-4617-acc9-46f48a17db68 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:42.034497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 227/465] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:19:41 2022] POST /identity/v3/users => generated 246 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:19:42.035398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd2978-b8c0-416d-a0a6-043b54a3bcf1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:42.122535 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:42.131468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a025f06-648b-4617-acc9-46f48a17db68 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:42.173049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e1a9a-2d4e-4967-a581-66b9350e1c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:42.173782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1a9a-2d4e-4967-a581-66b9350e1c44 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:42.174064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1a9a-2d4e-4967-a581-66b9350e1c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:42.174335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1a9a-2d4e-4967-a581-66b9350e1c44 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:42.175098 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e1a9a-2d4e-4967-a581-66b9350e1c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:42.175405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e1a9a-2d4e-4967-a581-66b9350e1c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:42.176893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e1a9a-2d4e-4967-a581-66b9350e1c44 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:42.191397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 239/466] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:42.193930 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5588-772b-4577-ba3d-1ecb91b19a22 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:42.194221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5588-772b-4577-ba3d-1ecb91b19a22 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:42.194498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5588-772b-4577-ba3d-1ecb91b19a22 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:42.195700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 240/467] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:42.196822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd2978-b8c0-416d-a0a6-043b54a3bcf1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:42.201510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc5588-772b-4577-ba3d-1ecb91b19a22 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:42.235919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:42.244581 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf566296-7948-41e4-930a-1da58889fb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:42.245922 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf566296-7948-41e4-930a-1da58889fb3a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:42.246422 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf566296-7948-41e4-930a-1da58889fb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:42.246901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf566296-7948-41e4-930a-1da58889fb3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:42.259548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf566296-7948-41e4-930a-1da58889fb3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:42.262411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf566296-7948-41e4-930a-1da58889fb3a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-cf566296-7948-41e4-930a-1da58889fb3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:42.280948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:42.281882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:42.282696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:42.282696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:42.282696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:42.284733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 228/468] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:42.286176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ebd02-3b06-4cd9-bac5-04561d720912 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:42.286663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ebd02-3b06-4cd9-bac5-04561d720912 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:42.287141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ebd02-3b06-4cd9-bac5-04561d720912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:42.310035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-904ebd02-3b06-4cd9-bac5-04561d720912 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:42.312209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-904ebd02-3b06-4cd9-bac5-04561d720912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:42.334621 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc5588-772b-4577-ba3d-1ecb91b19a22 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:42.371992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c842e-3ed9-4dc8-9b83-4de25f6b3d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:42.372637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c842e-3ed9-4dc8-9b83-4de25f6b3d2e None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:42.372917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c842e-3ed9-4dc8-9b83-4de25f6b3d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:42.373189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c842e-3ed9-4dc8-9b83-4de25f6b3d2e None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:42.373895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c842e-3ed9-4dc8-9b83-4de25f6b3d2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:42.375460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c842e-3ed9-4dc8-9b83-4de25f6b3d2e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:42.398123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 229/469] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:42.399135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ebd02-3b06-4cd9-bac5-04561d720912 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:42.439199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:42.468512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 241/470] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:19:42 2022] POST /identity/v3/users => generated 248 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:19:42.469876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c842e-3ed9-4dc8-9b83-4de25f6b3d2e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:42.518808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ebd02-3b06-4cd9-bac5-04561d720912 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:42.562633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbb055-426a-4d28-b293-74c17fd65bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:42.563576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbb055-426a-4d28-b293-74c17fd65bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:42.564583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbb055-426a-4d28-b293-74c17fd65bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:42.565069 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbb055-426a-4d28-b293-74c17fd65bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:42.565948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbb055-426a-4d28-b293-74c17fd65bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:42.566473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbb055-426a-4d28-b293-74c17fd65bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:42.568253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbb055-426a-4d28-b293-74c17fd65bd9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:42.583745 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 230/471] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:42.590028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbb055-426a-4d28-b293-74c17fd65bd9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:42.675098 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:42.715653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:42.723242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbb055-426a-4d28-b293-74c17fd65bd9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:42.725840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c842e-3ed9-4dc8-9b83-4de25f6b3d2e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:42.766537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86d2eb-1e6a-4584-a3b9-1444a80e1169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:42.767528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86d2eb-1e6a-4584-a3b9-1444a80e1169 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:42.768057 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86d2eb-1e6a-4584-a3b9-1444a80e1169 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:42.768524 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86d2eb-1e6a-4584-a3b9-1444a80e1169 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:42.769802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86d2eb-1e6a-4584-a3b9-1444a80e1169 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:42.771889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86d2eb-1e6a-4584-a3b9-1444a80e1169 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:42.794421 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4abc7-4832-4324-9668-171c6fc0687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:42.795173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4abc7-4832-4324-9668-171c6fc0687c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:42.795455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4abc7-4832-4324-9668-171c6fc0687c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:42.795723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4abc7-4832-4324-9668-171c6fc0687c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:42.809088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4abc7-4832-4324-9668-171c6fc0687c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:42.810653 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4abc7-4832-4324-9668-171c6fc0687c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-a2e4abc7-4832-4324-9668-171c6fc0687c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:42.830812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:42.831735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:42.833057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:42.833057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:42.833057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:42.833057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 242/472] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:42.838225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4abc7-4832-4324-9668-171c6fc0687c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:42.879363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 231/473] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:19:42 2022] POST /identity/v3/users => generated 248 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:19:42.925920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472eb5-d27b-4d79-94ac-7933e409e6ea None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:42.926801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472eb5-d27b-4d79-94ac-7933e409e6ea None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:42.927267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472eb5-d27b-4d79-94ac-7933e409e6ea None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:42.929785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 232/474] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:42.935867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43a6d1-4f2c-431d-976c-358e1c37221a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:42.936489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43a6d1-4f2c-431d-976c-358e1c37221a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:42.937030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43a6d1-4f2c-431d-976c-358e1c37221a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:42.963749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4abc7-4832-4324-9668-171c6fc0687c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:42.964100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-dc43a6d1-4f2c-431d-976c-358e1c37221a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:42.965525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-dc43a6d1-4f2c-431d-976c-358e1c37221a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:43.006677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c4a2a-3330-4a73-89c0-1c9235c1c399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:43.007345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c4a2a-3330-4a73-89c0-1c9235c1c399 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:43.007670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c4a2a-3330-4a73-89c0-1c9235c1c399 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:43.007971 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c4a2a-3330-4a73-89c0-1c9235c1c399 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:43.008678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c4a2a-3330-4a73-89c0-1c9235c1c399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:43.008980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c4a2a-3330-4a73-89c0-1c9235c1c399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:43.010378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c4a2a-3330-4a73-89c0-1c9235c1c399 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:43.012805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:43.038040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 243/475] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:43.044689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c4a2a-3330-4a73-89c0-1c9235c1c399 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:43.088008 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 233/476] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:43.112907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43a6d1-4f2c-431d-976c-358e1c37221a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:43.179472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c4a2a-3330-4a73-89c0-1c9235c1c399 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:43.216545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:43.220762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb289-c92c-45c5-8497-7d9c52b9becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:43.221921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb289-c92c-45c5-8497-7d9c52b9becf None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:43.222412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb289-c92c-45c5-8497-7d9c52b9becf None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:43.222875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb289-c92c-45c5-8497-7d9c52b9becf None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:43.223697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fb289-c92c-45c5-8497-7d9c52b9becf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:43.225375 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fb289-c92c-45c5-8497-7d9c52b9becf None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:43.236675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43a6d1-4f2c-431d-976c-358e1c37221a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:43.282593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371d75a-611c-48b6-a62d-a8186ed22205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:43.283258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d75a-611c-48b6-a62d-a8186ed22205 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:43.283539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d75a-611c-48b6-a62d-a8186ed22205 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:43.283813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d75a-611c-48b6-a62d-a8186ed22205 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:43.284573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371d75a-611c-48b6-a62d-a8186ed22205 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:43.285952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371d75a-611c-48b6-a62d-a8186ed22205 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:43.309476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 244/477] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:19:43 2022] POST /identity/v3/users => generated 249 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:19:43.325888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.sql.core [None req-a371d75a-611c-48b6-a62d-a8186ed22205 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 22:19:43.325888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 07 22:19:43.325888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [parameters: {'id': '5ffc6f2e00cd49e2b6093727c22852de', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 07 22:19:43.325888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80370) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-a371d75a-611c-48b6-a62d-a8186ed22205 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application conn.query(q) Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application result.read() Aug 07 22:19:43.343218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return ref.to_dict() Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 22:19:43.344147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application next(self.gen) Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application yield resource Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application next(self.gen) Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise self.value Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application session.commit() Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 22:19:43.345044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self.session.flush() Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application compat.raise_( Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 22:19:43.345913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 22:19:43.346912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application conn.query(q) Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application result.read() Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 22:19:43.347725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [parameters: {'id': '5ffc6f2e00cd49e2b6093727c22852de', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:43.348692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 07 22:19:43.349476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:43.352220 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 234/478] 10.222.0.30 () {60 vars in 1131 bytes} [Sun Aug 7 22:19:43 2022] POST /identity/v3/roles => generated 144 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:43.352356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb289-c92c-45c5-8497-7d9c52b9becf None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:43.447599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:43.509410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb289-c92c-45c5-8497-7d9c52b9becf None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:43.546437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdf21f-bcab-4334-95e8-c5211669d109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:43.547072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf21f-bcab-4334-95e8-c5211669d109 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:43.547347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf21f-bcab-4334-95e8-c5211669d109 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:43.547619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf21f-bcab-4334-95e8-c5211669d109 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:43.558743 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fdf21f-bcab-4334-95e8-c5211669d109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-11fdf21f-bcab-4334-95e8-c5211669d109 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:43.563970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:43.565982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:43.567298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 245/479] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:19:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:43.571065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371d75a-611c-48b6-a62d-a8186ed22205 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:43.686327 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371d75a-611c-48b6-a62d-a8186ed22205 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:43.722982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4636ffa-6e6a-410e-b486-48a60d9a64d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:43.723587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4636ffa-6e6a-410e-b486-48a60d9a64d5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:43.723864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4636ffa-6e6a-410e-b486-48a60d9a64d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:43.724164 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4636ffa-6e6a-410e-b486-48a60d9a64d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:43.724822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4636ffa-6e6a-410e-b486-48a60d9a64d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:43.725120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4636ffa-6e6a-410e-b486-48a60d9a64d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:43.726468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4636ffa-6e6a-410e-b486-48a60d9a64d5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:43.735986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 235/480] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:44.415437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ba815-8be9-40af-a4a8-c055638cb172 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:44.415984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ba815-8be9-40af-a4a8-c055638cb172 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:44.415984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ba815-8be9-40af-a4a8-c055638cb172 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:44.418125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 246/481] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:44.430574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bf0c8-b0b9-49b2-ad42-98c6d86b15a8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:44.430982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bf0c8-b0b9-49b2-ad42-98c6d86b15a8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:44.431186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bf0c8-b0b9-49b2-ad42-98c6d86b15a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:44.454407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-db4bf0c8-b0b9-49b2-ad42-98c6d86b15a8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:44.455609 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-db4bf0c8-b0b9-49b2-ad42-98c6d86b15a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:44.457980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:44.528341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 236/482] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:44.572658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ba815-8be9-40af-a4a8-c055638cb172 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:44.703538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ba815-8be9-40af-a4a8-c055638cb172 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:44.714913 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:44.743799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d945fa1-c595-46c8-97d4-01fa04ef972a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:44.744455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d945fa1-c595-46c8-97d4-01fa04ef972a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:44.744732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d945fa1-c595-46c8-97d4-01fa04ef972a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:44.745438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d945fa1-c595-46c8-97d4-01fa04ef972a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:44.761934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d945fa1-c595-46c8-97d4-01fa04ef972a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-6d945fa1-c595-46c8-97d4-01fa04ef972a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:44.766758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:44.767709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:44.768531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 247/483] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:44.773797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bf0c8-b0b9-49b2-ad42-98c6d86b15a8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:44.903812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bf0c8-b0b9-49b2-ad42-98c6d86b15a8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:44.944055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8490a-faa1-4f66-8d4d-19d31f7a834c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:44.944676 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8490a-faa1-4f66-8d4d-19d31f7a834c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:44.944955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8490a-faa1-4f66-8d4d-19d31f7a834c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:44.945228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8490a-faa1-4f66-8d4d-19d31f7a834c None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:44.945889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8490a-faa1-4f66-8d4d-19d31f7a834c None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:44.946192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8490a-faa1-4f66-8d4d-19d31f7a834c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:44.947548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8490a-faa1-4f66-8d4d-19d31f7a834c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:44.956236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:44.960303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 237/484] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:44.969104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d945fa1-c595-46c8-97d4-01fa04ef972a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:45.112933 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d945fa1-c595-46c8-97d4-01fa04ef972a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:45.150383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a912ce7-89d6-426e-8c00-8764e09ade0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:45.151073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a912ce7-89d6-426e-8c00-8764e09ade0f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:45.151375 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a912ce7-89d6-426e-8c00-8764e09ade0f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:45.151766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a912ce7-89d6-426e-8c00-8764e09ade0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:45.162636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a912ce7-89d6-426e-8c00-8764e09ade0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:45.164155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a912ce7-89d6-426e-8c00-8764e09ade0f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-1a912ce7-89d6-426e-8c00-8764e09ade0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:45.188575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:45.189544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:45.190412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:45.190412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:45.190412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:45.190412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 248/485] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:45.196833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8490a-faa1-4f66-8d4d-19d31f7a834c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:45.216736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8272060-9b3e-489d-a90b-0633e169ed5b None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:45.216924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8272060-9b3e-489d-a90b-0633e169ed5b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:45.217199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8272060-9b3e-489d-a90b-0633e169ed5b None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:45.218767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 249/486] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:45.224280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21aaa8-fce4-49a9-8de8-09d5b209cf8f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:45.224575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21aaa8-fce4-49a9-8de8-09d5b209cf8f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:45.224824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21aaa8-fce4-49a9-8de8-09d5b209cf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:45.255923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6b21aaa8-fce4-49a9-8de8-09d5b209cf8f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:45.257167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6b21aaa8-fce4-49a9-8de8-09d5b209cf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:45.292346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:45.383924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 250/487] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:45.387736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8490a-faa1-4f66-8d4d-19d31f7a834c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:45.419059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21aaa8-fce4-49a9-8de8-09d5b209cf8f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:45.436851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e52ad6-455f-4d0f-81e6-01976f051a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:45.437472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52ad6-455f-4d0f-81e6-01976f051a7f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:45.437747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52ad6-455f-4d0f-81e6-01976f051a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:45.438020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52ad6-455f-4d0f-81e6-01976f051a7f None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:45.438667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e52ad6-455f-4d0f-81e6-01976f051a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:45.438973 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e52ad6-455f-4d0f-81e6-01976f051a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:45.441352 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e52ad6-455f-4d0f-81e6-01976f051a7f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:45.468191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 238/488] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:45.472164 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e52ad6-455f-4d0f-81e6-01976f051a7f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:45.583191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21aaa8-fce4-49a9-8de8-09d5b209cf8f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:45.597163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:45.647503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da2c7d-5235-4b12-8ca1-44c4cb77c4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:45.648202 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da2c7d-5235-4b12-8ca1-44c4cb77c4ba None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:45.648482 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da2c7d-5235-4b12-8ca1-44c4cb77c4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:45.648761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da2c7d-5235-4b12-8ca1-44c4cb77c4ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:45.670693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da2c7d-5235-4b12-8ca1-44c4cb77c4ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:45.677339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e52ad6-455f-4d0f-81e6-01976f051a7f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b1da2c7d-5235-4b12-8ca1-44c4cb77c4ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:45.679296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:45.680246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:45.681041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 251/489] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:45.689073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da2c7d-5235-4b12-8ca1-44c4cb77c4ba None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:45.708054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:45.764750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920c819-52f8-475d-abaf-0cc7a87020e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:45.765411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920c819-52f8-475d-abaf-0cc7a87020e4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:45.765697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920c819-52f8-475d-abaf-0cc7a87020e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:45.765980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920c819-52f8-475d-abaf-0cc7a87020e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:45.785150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920c819-52f8-475d-abaf-0cc7a87020e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:45.786612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920c819-52f8-475d-abaf-0cc7a87020e4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:45.807230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da2c7d-5235-4b12-8ca1-44c4cb77c4ba None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-1920c819-52f8-475d-abaf-0cc7a87020e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:19:45.811490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:45.812395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:45.813310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:45.813310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:45.813310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:19:45.813310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:19:45.813310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:19:45.813310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:19:45.813310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:19:45.813310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:19:45.813310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:19:45.813310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:45.820902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 239/490] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 22:19:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:45.826676 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920c819-52f8-475d-abaf-0cc7a87020e4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:45.874640 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce5a9d-cc9a-4743-9f49-0ed11f37f387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:45.875607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce5a9d-cc9a-4743-9f49-0ed11f37f387 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:45.876992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce5a9d-cc9a-4743-9f49-0ed11f37f387 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:45.877498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce5a9d-cc9a-4743-9f49-0ed11f37f387 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:45.879572 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce5a9d-cc9a-4743-9f49-0ed11f37f387 None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:45.881313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce5a9d-cc9a-4743-9f49-0ed11f37f387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:45.883915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce5a9d-cc9a-4743-9f49-0ed11f37f387 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:45.910348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 252/491] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:45.925586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce5a9d-cc9a-4743-9f49-0ed11f37f387 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:46.003155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920c819-52f8-475d-abaf-0cc7a87020e4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:46.024786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:46.025646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:46.041054 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598463f-da99-4e1e-a8c4-afa9626292ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:46.042570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598463f-da99-4e1e-a8c4-afa9626292ac None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.043364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598463f-da99-4e1e-a8c4-afa9626292ac None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.043877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598463f-da99-4e1e-a8c4-afa9626292ac None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.044858 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598463f-da99-4e1e-a8c4-afa9626292ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:46.045368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598463f-da99-4e1e-a8c4-afa9626292ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:46.047086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598463f-da99-4e1e-a8c4-afa9626292ac None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:46.051664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce5a9d-cc9a-4743-9f49-0ed11f37f387 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:46.088172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435a720-9849-4360-9ce4-7f3f4b7c3ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:46.088819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435a720-9849-4360-9ce4-7f3f4b7c3ecb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.089901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 240/492] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 22:19:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:46.090092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435a720-9849-4360-9ce4-7f3f4b7c3ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.090092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435a720-9849-4360-9ce4-7f3f4b7c3ecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.095580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b005976-d4f3-4c32-97f9-691a40408499 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.095874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b005976-d4f3-4c32-97f9-691a40408499 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.096182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b005976-d4f3-4c32-97f9-691a40408499 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.097573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 241/493] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:46.101643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9bd49-79a8-4387-9817-fbe77b72a4eb None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.101776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435a720-9849-4360-9ce4-7f3f4b7c3ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:46.102256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9bd49-79a8-4387-9817-fbe77b72a4eb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.102731 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9bd49-79a8-4387-9817-fbe77b72a4eb None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.103223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435a720-9849-4360-9ce4-7f3f4b7c3ecb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:46.105354 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 242/494] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:46.106884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9bd49-79a8-4387-9817-fbe77b72a4eb None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-9435a720-9849-4360-9ce4-7f3f4b7c3ecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:46.118236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:46.119198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:46.125423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:46.125423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:46.125423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:46.125423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 253/495] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:46.125423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601febb-cf11-4e1e-8cf8-db15406b3fba None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.125423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601febb-cf11-4e1e-8cf8-db15406b3fba None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.125423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601febb-cf11-4e1e-8cf8-db15406b3fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.142512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4601febb-cf11-4e1e-8cf8-db15406b3fba None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:46.143713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4601febb-cf11-4e1e-8cf8-db15406b3fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:46.211736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 254/496] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:46.213093 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a86a5-f365-425f-b14e-5f0bc6ca1f35 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.213384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a86a5-f365-425f-b14e-5f0bc6ca1f35 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.213655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a86a5-f365-425f-b14e-5f0bc6ca1f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.231502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9bd49-79a8-4387-9817-fbe77b72a4eb None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:46.232040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-150a86a5-f365-425f-b14e-5f0bc6ca1f35 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:46.233264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-150a86a5-f365-425f-b14e-5f0bc6ca1f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:46.266129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ff255-4e4f-430b-b416-d181847ce92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:46.266733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ff255-4e4f-430b-b416-d181847ce92f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.267011 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ff255-4e4f-430b-b416-d181847ce92f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.267284 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ff255-4e4f-430b-b416-d181847ce92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.284601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:46.300625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 255/497] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:46.301441 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a86a5-f365-425f-b14e-5f0bc6ca1f35 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:46.314439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:46.336799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ff255-4e4f-430b-b416-d181847ce92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:46.338258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ff255-4e4f-430b-b416-d181847ce92f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:46.400405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a86a5-f365-425f-b14e-5f0bc6ca1f35 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:46.435059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecb662-bfd0-438a-9924-65582c081bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:46.435678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecb662-bfd0-438a-9924-65582c081bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.435955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecb662-bfd0-438a-9924-65582c081bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.436258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecb662-bfd0-438a-9924-65582c081bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.436900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecb662-bfd0-438a-9924-65582c081bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:46.437208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecb662-bfd0-438a-9924-65582c081bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:46.438611 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecb662-bfd0-438a-9924-65582c081bb8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:46.438997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 243/498] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:46.440300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951434ad-8edb-45c1-95b1-f30062cd0e3b None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.440589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951434ad-8edb-45c1-95b1-f30062cd0e3b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.441215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951434ad-8edb-45c1-95b1-f30062cd0e3b None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.442290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 244/499] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:46.442789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951434ad-8edb-45c1-95b1-f30062cd0e3b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:46.453129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 256/500] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:46.453947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecb662-bfd0-438a-9924-65582c081bb8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:46.500108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:46.526635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:46.548566 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951434ad-8edb-45c1-95b1-f30062cd0e3b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:46.552776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecb662-bfd0-438a-9924-65582c081bb8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:46.582832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5f3ac-c94e-4fe9-960a-2abcfb125452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:46.583457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5f3ac-c94e-4fe9-960a-2abcfb125452 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.583737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5f3ac-c94e-4fe9-960a-2abcfb125452 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.584036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5f3ac-c94e-4fe9-960a-2abcfb125452 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.587837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38998d6-a455-4f2c-a0eb-6fb021e1a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:46.588491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38998d6-a455-4f2c-a0eb-6fb021e1a204 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.588795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38998d6-a455-4f2c-a0eb-6fb021e1a204 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.589220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38998d6-a455-4f2c-a0eb-6fb021e1a204 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.594166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5f3ac-c94e-4fe9-960a-2abcfb125452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-7de5f3ac-c94e-4fe9-960a-2abcfb125452 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:46.598938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:46.599946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:46.600800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:46.601399 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38998d6-a455-4f2c-a0eb-6fb021e1a204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:46.601467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 245/501] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:46.601467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5f3ac-c94e-4fe9-960a-2abcfb125452 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-d38998d6-a455-4f2c-a0eb-6fb021e1a204 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:46.604602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:46.605502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:46.606318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 257/502] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:46.606981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fadc30-11d2-4b9f-98dc-17cb40ac0650 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.607272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fadc30-11d2-4b9f-98dc-17cb40ac0650 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.607559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fadc30-11d2-4b9f-98dc-17cb40ac0650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.626175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a0fadc30-11d2-4b9f-98dc-17cb40ac0650 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:46.627392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a0fadc30-11d2-4b9f-98dc-17cb40ac0650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:46.694524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 258/503] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:46.695598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fadc30-11d2-4b9f-98dc-17cb40ac0650 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:46.716241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5f3ac-c94e-4fe9-960a-2abcfb125452 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:46.728723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:46.751265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1a7e3-f378-42a2-89f6-e12ac6ee9f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:46.751895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1a7e3-f378-42a2-89f6-e12ac6ee9f99 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.752216 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1a7e3-f378-42a2-89f6-e12ac6ee9f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.752521 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1a7e3-f378-42a2-89f6-e12ac6ee9f99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.759046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:46.763700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1a7e3-f378-42a2-89f6-e12ac6ee9f99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:46.765176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1a7e3-f378-42a2-89f6-e12ac6ee9f99 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-5ba1a7e3-f378-42a2-89f6-e12ac6ee9f99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:46.779663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:46.780662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:46.781600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:46.781600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:46.781600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:46.781600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 246/504] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:46.782080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1a7e3-f378-42a2-89f6-e12ac6ee9f99 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:46.798503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fadc30-11d2-4b9f-98dc-17cb40ac0650 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:46.832624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577ead4-4414-4853-9a24-f0f6ce72cfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:46.833180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577ead4-4414-4853-9a24-f0f6ce72cfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.833461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577ead4-4414-4853-9a24-f0f6ce72cfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.833732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577ead4-4414-4853-9a24-f0f6ce72cfc9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.840757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577ead4-4414-4853-9a24-f0f6ce72cfc9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:46.842201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577ead4-4414-4853-9a24-f0f6ce72cfc9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-1577ead4-4414-4853-9a24-f0f6ce72cfc9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:19:46.855298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:46.856213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:46.857102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:46.857102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:46.857102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:19:46.857102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:19:46.857102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:19:46.857102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:19:46.857102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:19:46.857102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:19:46.857102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 22:19:46.857102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:46.857102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 259/505] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:46.857658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577ead4-4414-4853-9a24-f0f6ce72cfc9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:46.886221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1a7e3-f378-42a2-89f6-e12ac6ee9f99 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:46.919642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977bfdd-977e-4e7f-a30b-560e7449c4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:46.920273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977bfdd-977e-4e7f-a30b-560e7449c4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.920553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977bfdd-977e-4e7f-a30b-560e7449c4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.920827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977bfdd-977e-4e7f-a30b-560e7449c4f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.921486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977bfdd-977e-4e7f-a30b-560e7449c4f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:46.921789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977bfdd-977e-4e7f-a30b-560e7449c4f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:46.923139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977bfdd-977e-4e7f-a30b-560e7449c4f9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:46.930875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 247/506] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:46.931646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977bfdd-977e-4e7f-a30b-560e7449c4f9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:46.951675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:46.957298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577ead4-4414-4853-9a24-f0f6ce72cfc9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:46.988785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:46.991553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0924989-5219-4343-8f33-2c1335c51caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:46.992195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0924989-5219-4343-8f33-2c1335c51caf None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:46.992473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0924989-5219-4343-8f33-2c1335c51caf None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:46.992745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0924989-5219-4343-8f33-2c1335c51caf None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:46.993409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0924989-5219-4343-8f33-2c1335c51caf None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:46.993711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0924989-5219-4343-8f33-2c1335c51caf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:46.996171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0924989-5219-4343-8f33-2c1335c51caf None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.003992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 260/507] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:47.004796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0924989-5219-4343-8f33-2c1335c51caf None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:47.040328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977bfdd-977e-4e7f-a30b-560e7449c4f9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:47.073826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e73193-b116-4eab-a899-4a8e3ff6eb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:47.074447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e73193-b116-4eab-a899-4a8e3ff6eb22 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:47.074730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e73193-b116-4eab-a899-4a8e3ff6eb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:47.075007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e73193-b116-4eab-a899-4a8e3ff6eb22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:47.084864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e73193-b116-4eab-a899-4a8e3ff6eb22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:47.086280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e73193-b116-4eab-a899-4a8e3ff6eb22 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f9e73193-b116-4eab-a899-4a8e3ff6eb22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:47.101167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:47.102218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:47.103091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:47.103091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:47.103091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:47.103258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0924989-5219-4343-8f33-2c1335c51caf None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:47.103337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 248/508] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:47.104413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e73193-b116-4eab-a899-4a8e3ff6eb22 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:47.135464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcbbb8-a089-4661-af19-101e8ef9d764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:47.136235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcbbb8-a089-4661-af19-101e8ef9d764 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:47.136515 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcbbb8-a089-4661-af19-101e8ef9d764 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:47.136788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcbbb8-a089-4661-af19-101e8ef9d764 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:47.137423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fcbbb8-a089-4661-af19-101e8ef9d764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:47.137725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fcbbb8-a089-4661-af19-101e8ef9d764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:47.139091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fcbbb8-a089-4661-af19-101e8ef9d764 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.150477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 261/509] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:47.151763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcbbb8-a089-4661-af19-101e8ef9d764 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:47.171138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:47.207424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e73193-b116-4eab-a899-4a8e3ff6eb22 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:47.209808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:47.241828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a715e4-3321-4c4a-a44e-46ccb0ccbf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:47.242423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a715e4-3321-4c4a-a44e-46ccb0ccbf70 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:47.242699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a715e4-3321-4c4a-a44e-46ccb0ccbf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:47.242973 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a715e4-3321-4c4a-a44e-46ccb0ccbf70 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:47.243700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a715e4-3321-4c4a-a44e-46ccb0ccbf70 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:47.244005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a715e4-3321-4c4a-a44e-46ccb0ccbf70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:47.245826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a715e4-3321-4c4a-a44e-46ccb0ccbf70 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.255174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcbbb8-a089-4661-af19-101e8ef9d764 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:47.262477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 249/510] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 22:19:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:47.263267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a715e4-3321-4c4a-a44e-46ccb0ccbf70 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:47.294738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b5630-f9c4-4638-812b-53affd4529d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:47.295364 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b5630-f9c4-4638-812b-53affd4529d7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:47.295640 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b5630-f9c4-4638-812b-53affd4529d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:47.295913 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b5630-f9c4-4638-812b-53affd4529d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:47.305819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b5630-f9c4-4638-812b-53affd4529d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:47.307270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b5630-f9c4-4638-812b-53affd4529d7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ac5b5630-f9c4-4638-812b-53affd4529d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:47.321404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:47.322459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:47.323522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:47.323522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:47.323522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:47.323522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 262/511] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:47.323522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b5630-f9c4-4638-812b-53affd4529d7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:47.366522 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a715e4-3321-4c4a-a44e-46ccb0ccbf70 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:47.391849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:47.400133 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54567ff1-0e56-4071-9c98-8a887377e047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:47.400752 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54567ff1-0e56-4071-9c98-8a887377e047 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:47.401022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54567ff1-0e56-4071-9c98-8a887377e047 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:47.401942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54567ff1-0e56-4071-9c98-8a887377e047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:47.411678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54567ff1-0e56-4071-9c98-8a887377e047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:47.413167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54567ff1-0e56-4071-9c98-8a887377e047 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.422283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b5630-f9c4-4638-812b-53affd4529d7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-54567ff1-0e56-4071-9c98-8a887377e047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:47.427383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:47.428440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:47.429487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:47.429487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:47.429487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:47.429813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 250/512] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:47.430614 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54567ff1-0e56-4071-9c98-8a887377e047 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:47.446001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:47.457133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fae13b-e0cd-449c-859b-d5d6df0bb8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:47.465832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fae13b-e0cd-449c-859b-d5d6df0bb8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:47.465832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fae13b-e0cd-449c-859b-d5d6df0bb8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:47.465832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fae13b-e0cd-449c-859b-d5d6df0bb8a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:47.465832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fae13b-e0cd-449c-859b-d5d6df0bb8a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:47.465832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fae13b-e0cd-449c-859b-d5d6df0bb8a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:47.465832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fae13b-e0cd-449c-859b-d5d6df0bb8a5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.473268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 263/513] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:47.474004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fae13b-e0cd-449c-859b-d5d6df0bb8a5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:47.531087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54567ff1-0e56-4071-9c98-8a887377e047 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:47.567833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb432c-70cf-46da-b991-a7a542496ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:47.568486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb432c-70cf-46da-b991-a7a542496ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:47.568762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb432c-70cf-46da-b991-a7a542496ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:47.569032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb432c-70cf-46da-b991-a7a542496ce9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:47.571162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fae13b-e0cd-449c-859b-d5d6df0bb8a5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:47.575745 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb432c-70cf-46da-b991-a7a542496ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:47.577240 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb432c-70cf-46da-b991-a7a542496ce9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-2fbb432c-70cf-46da-b991-a7a542496ce9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:19:47.588552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:47.589405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:47.590256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:19:47.590256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:19:47.590256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:19:47.590256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:19:47.590256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:19:47.590256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:19:47.590256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:47.590256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:47.590256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 251/514] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:47.590717 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb432c-70cf-46da-b991-a7a542496ce9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:47.603384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:47.609620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d5373-0332-49c2-b90b-1b0aa3628fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:47.613952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d5373-0332-49c2-b90b-1b0aa3628fbc None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:47.613952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d5373-0332-49c2-b90b-1b0aa3628fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:47.613952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d5373-0332-49c2-b90b-1b0aa3628fbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:47.670232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:47.678641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d5373-0332-49c2-b90b-1b0aa3628fbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:47.680132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d5373-0332-49c2-b90b-1b0aa3628fbc None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.694455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb432c-70cf-46da-b991-a7a542496ce9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:47.727420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079c2f5-42d7-41fc-aa4b-553f992b92aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:47.728033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079c2f5-42d7-41fc-aa4b-553f992b92aa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:47.728338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079c2f5-42d7-41fc-aa4b-553f992b92aa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:47.728614 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079c2f5-42d7-41fc-aa4b-553f992b92aa None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:47.729254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079c2f5-42d7-41fc-aa4b-553f992b92aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:47.730105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079c2f5-42d7-41fc-aa4b-553f992b92aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:47.731576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079c2f5-42d7-41fc-aa4b-553f992b92aa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.743348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 252/515] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:47.744154 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079c2f5-42d7-41fc-aa4b-553f992b92aa None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:47.782735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 264/516] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:47.783600 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d5373-0332-49c2-b90b-1b0aa3628fbc None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:47.812986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:47.843973 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079c2f5-42d7-41fc-aa4b-553f992b92aa None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:47.878235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdba16-2514-439d-8e1d-ed27283b52bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:47.878838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdba16-2514-439d-8e1d-ed27283b52bc None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:47.879113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdba16-2514-439d-8e1d-ed27283b52bc None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:47.879392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdba16-2514-439d-8e1d-ed27283b52bc None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:47.880050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdba16-2514-439d-8e1d-ed27283b52bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:47.880373 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdba16-2514-439d-8e1d-ed27283b52bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:47.881727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdba16-2514-439d-8e1d-ed27283b52bc None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.882094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d5373-0332-49c2-b90b-1b0aa3628fbc None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:47.883090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:47.894152 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 253/517] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:47.894968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdba16-2514-439d-8e1d-ed27283b52bc None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:47.918211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c1623-0def-43d8-a7e0-ed7cc5fa4362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:47.918817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c1623-0def-43d8-a7e0-ed7cc5fa4362 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:47.919103 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c1623-0def-43d8-a7e0-ed7cc5fa4362 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:47.919400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c1623-0def-43d8-a7e0-ed7cc5fa4362 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:47.920059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c1623-0def-43d8-a7e0-ed7cc5fa4362 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:47.921554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c1623-0def-43d8-a7e0-ed7cc5fa4362 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:47.991327 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdba16-2514-439d-8e1d-ed27283b52bc None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:48.016951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 265/518] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:19:47 2022] POST /identity/v3/users => generated 247 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:19:48.017800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c1623-0def-43d8-a7e0-ed7cc5fa4362 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:48.034626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636e0b7-a6ab-4c4c-90c3-c65bd0c87780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:48.035249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636e0b7-a6ab-4c4c-90c3-c65bd0c87780 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:48.035525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636e0b7-a6ab-4c4c-90c3-c65bd0c87780 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:48.035801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636e0b7-a6ab-4c4c-90c3-c65bd0c87780 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:48.036543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636e0b7-a6ab-4c4c-90c3-c65bd0c87780 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:48.036841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636e0b7-a6ab-4c4c-90c3-c65bd0c87780 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:48.038725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636e0b7-a6ab-4c4c-90c3-c65bd0c87780 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:48.065346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 254/519] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:19:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:48.065346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636e0b7-a6ab-4c4c-90c3-c65bd0c87780 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:48.116577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:48.128685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:48.134054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c1623-0def-43d8-a7e0-ed7cc5fa4362 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:48.175122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d1ea8-133e-46ce-b0b3-d3d874a3dd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:48.175683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d1ea8-133e-46ce-b0b3-d3d874a3dd2f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:48.175957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d1ea8-133e-46ce-b0b3-d3d874a3dd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:48.176254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d1ea8-133e-46ce-b0b3-d3d874a3dd2f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:48.179866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636e0b7-a6ab-4c4c-90c3-c65bd0c87780 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:48.185532 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d1ea8-133e-46ce-b0b3-d3d874a3dd2f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:48.186993 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d1ea8-133e-46ce-b0b3-d3d874a3dd2f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-a84d1ea8-133e-46ce-b0b3-d3d874a3dd2f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:19:48.201726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:48.202854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:48.204111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:48.204111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:48.204111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:19:48.204111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:19:48.204111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:19:48.204111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:19:48.204111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:19:48.204111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:19:48.204111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 22:19:48.204111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:48.204111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 266/520] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:48.204684 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d1ea8-133e-46ce-b0b3-d3d874a3dd2f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:48.223835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52c84d-3d45-4184-b6f3-7147ca9a8ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:48.224501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52c84d-3d45-4184-b6f3-7147ca9a8ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:48.224772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52c84d-3d45-4184-b6f3-7147ca9a8ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:48.225040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52c84d-3d45-4184-b6f3-7147ca9a8ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:48.237070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52c84d-3d45-4184-b6f3-7147ca9a8ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:48.238458 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52c84d-3d45-4184-b6f3-7147ca9a8ba9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-da52c84d-3d45-4184-b6f3-7147ca9a8ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:48.253196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:48.254093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:48.254930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:48.254930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:48.254930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:48.254930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 255/521] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:48.255138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52c84d-3d45-4184-b6f3-7147ca9a8ba9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:48.319402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d1ea8-133e-46ce-b0b3-d3d874a3dd2f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:48.356524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28a9e0-597a-49ef-8958-28406f8c2427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:48.357384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28a9e0-597a-49ef-8958-28406f8c2427 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:48.357813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28a9e0-597a-49ef-8958-28406f8c2427 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:48.358092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28a9e0-597a-49ef-8958-28406f8c2427 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:48.360421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52c84d-3d45-4184-b6f3-7147ca9a8ba9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:48.365919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28a9e0-597a-49ef-8958-28406f8c2427 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:48.367442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28a9e0-597a-49ef-8958-28406f8c2427 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:48.368256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-9a28a9e0-597a-49ef-8958-28406f8c2427 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:19:48.379934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:48.391824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 267/522] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28a9e0-597a-49ef-8958-28406f8c2427 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:48.393133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:48.404203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34003633-b9fb-48a4-867d-6370c6c33277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:48.404802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34003633-b9fb-48a4-867d-6370c6c33277 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:48.405078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34003633-b9fb-48a4-867d-6370c6c33277 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:48.405402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34003633-b9fb-48a4-867d-6370c6c33277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:48.406131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34003633-b9fb-48a4-867d-6370c6c33277 None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=8effbd0670ef41f59b46feb9f0997554) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:48.406486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34003633-b9fb-48a4-867d-6370c6c33277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:48.407930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34003633-b9fb-48a4-867d-6370c6c33277 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:48.416165 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 256/523] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:48 2022] GET /identity/v3/role_assignments?user.id=8effbd0670ef41f59b46feb9f0997554&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:48.417520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34003633-b9fb-48a4-867d-6370c6c33277 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:48.519435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28a9e0-597a-49ef-8958-28406f8c2427 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:48.538663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34003633-b9fb-48a4-867d-6370c6c33277 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:48.565580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cd697-a138-4f84-998f-ce0e4295f79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:48.566587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cd697-a138-4f84-998f-ce0e4295f79e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:48.567059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cd697-a138-4f84-998f-ce0e4295f79e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:48.567522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cd697-a138-4f84-998f-ce0e4295f79e None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:48.568554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cd697-a138-4f84-998f-ce0e4295f79e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:48.569090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cd697-a138-4f84-998f-ce0e4295f79e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:48.570776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cd697-a138-4f84-998f-ce0e4295f79e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:48.582860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03e36f-2e6b-469e-a8f8-04b76a6358d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:48.583468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03e36f-2e6b-469e-a8f8-04b76a6358d8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:48.583749 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03e36f-2e6b-469e-a8f8-04b76a6358d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:48.584042 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03e36f-2e6b-469e-a8f8-04b76a6358d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:48.584703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03e36f-2e6b-469e-a8f8-04b76a6358d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:48.585990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03e36f-2e6b-469e-a8f8-04b76a6358d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:48.587379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03e36f-2e6b-469e-a8f8-04b76a6358d8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:48.600612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 257/524] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:48.602419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03e36f-2e6b-469e-a8f8-04b76a6358d8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:48.616219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 268/525] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 22:19:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:48.616989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cd697-a138-4f84-998f-ce0e4295f79e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:48.639800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:48.688627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:48.731071 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cd697-a138-4f84-998f-ce0e4295f79e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:48.734812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03e36f-2e6b-469e-a8f8-04b76a6358d8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:48.773986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cd592-723d-4cc2-8444-08ad46ae819c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:48.774839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cd592-723d-4cc2-8444-08ad46ae819c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:48.775254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cd592-723d-4cc2-8444-08ad46ae819c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:48.775698 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cd592-723d-4cc2-8444-08ad46ae819c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:48.784207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cd592-723d-4cc2-8444-08ad46ae819c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:48.785940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cd592-723d-4cc2-8444-08ad46ae819c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:48.787000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8b00f-1db7-4fee-b86b-9410bd3c69ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:48.787704 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8b00f-1db7-4fee-b86b-9410bd3c69ec None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:48.788108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8b00f-1db7-4fee-b86b-9410bd3c69ec None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:48.788590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8b00f-1db7-4fee-b86b-9410bd3c69ec None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:48.789691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8b00f-1db7-4fee-b86b-9410bd3c69ec None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:48.790067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8b00f-1db7-4fee-b86b-9410bd3c69ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:48.791557 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8b00f-1db7-4fee-b86b-9410bd3c69ec None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-526cd592-723d-4cc2-8444-08ad46ae819c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:19:48.800585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:48.801513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:48.802350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:19:48.802350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:19:48.802350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:19:48.802350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:19:48.802350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:19:48.802350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:19:48.802350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:48.802350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:48.804779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 269/526] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:48.805852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cd592-723d-4cc2-8444-08ad46ae819c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:48.818085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 258/527] 10.222.0.30 () {56 vars in 1139 bytes} [Sun Aug 7 22:19:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:48.819139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8b00f-1db7-4fee-b86b-9410bd3c69ec None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:48.912924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cd592-723d-4cc2-8444-08ad46ae819c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:48.920816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:48.935613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8b00f-1db7-4fee-b86b-9410bd3c69ec None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:48.944384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:48.949925 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c338517-ca40-4a00-afd7-8b7c9de3e6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:48.950610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338517-ca40-4a00-afd7-8b7c9de3e6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:48.950987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338517-ca40-4a00-afd7-8b7c9de3e6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:48.951355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338517-ca40-4a00-afd7-8b7c9de3e6f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:48.973632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465cce4-22ae-4c9c-9ed2-f016f8de3c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:48.977000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465cce4-22ae-4c9c-9ed2-f016f8de3c57 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:48.977000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465cce4-22ae-4c9c-9ed2-f016f8de3c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:48.977000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465cce4-22ae-4c9c-9ed2-f016f8de3c57 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:48.977000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465cce4-22ae-4c9c-9ed2-f016f8de3c57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:48.977000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465cce4-22ae-4c9c-9ed2-f016f8de3c57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:48.977481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465cce4-22ae-4c9c-9ed2-f016f8de3c57 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:48.990583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 259/528] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:19:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:48.991483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465cce4-22ae-4c9c-9ed2-f016f8de3c57 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:49.022126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c338517-ca40-4a00-afd7-8b7c9de3e6f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:49.023681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c338517-ca40-4a00-afd7-8b7c9de3e6f8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:49.095129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465cce4-22ae-4c9c-9ed2-f016f8de3c57 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:49.130267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 270/529] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:49.130963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f13382-2b04-48b4-839a-bca09a12e30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:49.131340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c338517-ca40-4a00-afd7-8b7c9de3e6f8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:49.131563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f13382-2b04-48b4-839a-bca09a12e30a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:49.131842 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f13382-2b04-48b4-839a-bca09a12e30a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:49.132142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f13382-2b04-48b4-839a-bca09a12e30a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:49.151476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:49.160410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:49.224177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f13382-2b04-48b4-839a-bca09a12e30a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:49.226441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f13382-2b04-48b4-839a-bca09a12e30a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:49.256713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c338517-ca40-4a00-afd7-8b7c9de3e6f8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:49.302204 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98f622-ed07-452f-9700-ec8b787898b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:49.303160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98f622-ed07-452f-9700-ec8b787898b8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:49.303510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98f622-ed07-452f-9700-ec8b787898b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:49.303870 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98f622-ed07-452f-9700-ec8b787898b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:49.304847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98f622-ed07-452f-9700-ec8b787898b8 None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=cfb2129c006f4ddeba7d128a8d43d5de) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:49.305227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98f622-ed07-452f-9700-ec8b787898b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:49.307312 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98f622-ed07-452f-9700-ec8b787898b8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:49.317793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 271/530] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:49 2022] GET /identity/v3/role_assignments?user.id=cfb2129c006f4ddeba7d128a8d43d5de&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:49.318781 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98f622-ed07-452f-9700-ec8b787898b8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:49.351593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 260/531] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:49.357375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f13382-2b04-48b4-839a-bca09a12e30a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:49.424246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:49.438340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98f622-ed07-452f-9700-ec8b787898b8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:49.479098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:49.484345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bf3c1-8fd2-43cd-babe-8d437f2c4377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:49.485344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bf3c1-8fd2-43cd-babe-8d437f2c4377 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:49.485822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bf3c1-8fd2-43cd-babe-8d437f2c4377 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:49.486266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bf3c1-8fd2-43cd-babe-8d437f2c4377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:49.502243 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bf3c1-8fd2-43cd-babe-8d437f2c4377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:49.504092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bf3c1-8fd2-43cd-babe-8d437f2c4377 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-4b1bf3c1-8fd2-43cd-babe-8d437f2c4377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:49.520179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:49.521552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:49.522419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:49.522419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:49.522419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:49.524322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 272/532] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:49.535059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bf3c1-8fd2-43cd-babe-8d437f2c4377 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:49.555274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f13382-2b04-48b4-839a-bca09a12e30a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:49.593376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588fce3-4d6b-4d92-af3e-60a5e65e5ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:49.594013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588fce3-4d6b-4d92-af3e-60a5e65e5ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:49.594293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588fce3-4d6b-4d92-af3e-60a5e65e5ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:49.594566 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588fce3-4d6b-4d92-af3e-60a5e65e5ad0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:49.606868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588fce3-4d6b-4d92-af3e-60a5e65e5ad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:49.608288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588fce3-4d6b-4d92-af3e-60a5e65e5ad0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-9588fce3-4d6b-4d92-af3e-60a5e65e5ad0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:49.623565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:49.624737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:49.625766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:49.625766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:49.625766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:49.632702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 261/533] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:49.638677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588fce3-4d6b-4d92-af3e-60a5e65e5ad0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:49.668962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bf3c1-8fd2-43cd-babe-8d437f2c4377 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:49.727763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b594f9-9c29-4397-bfa7-5d3995f5b9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:49.728709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b594f9-9c29-4397-bfa7-5d3995f5b9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:49.729134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b594f9-9c29-4397-bfa7-5d3995f5b9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:49.730174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b594f9-9c29-4397-bfa7-5d3995f5b9c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:49.731233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b594f9-9c29-4397-bfa7-5d3995f5b9c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:49.731896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b594f9-9c29-4397-bfa7-5d3995f5b9c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:49.735049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b594f9-9c29-4397-bfa7-5d3995f5b9c5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:49.752570 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:49.757199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 273/534] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:49.765404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b594f9-9c29-4397-bfa7-5d3995f5b9c5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:49.805019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588fce3-4d6b-4d92-af3e-60a5e65e5ad0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:49.808054 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:49.841939 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2d3c1-dd2b-4275-8f0f-4221d9b93709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:49.842641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d3c1-dd2b-4275-8f0f-4221d9b93709 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:49.842997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d3c1-dd2b-4275-8f0f-4221d9b93709 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:49.843339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d3c1-dd2b-4275-8f0f-4221d9b93709 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:49.844090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2d3c1-dd2b-4275-8f0f-4221d9b93709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:49.844462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2d3c1-dd2b-4275-8f0f-4221d9b93709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:49.845908 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2d3c1-dd2b-4275-8f0f-4221d9b93709 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:49.858994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 262/535] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:49.865551 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2d3c1-dd2b-4275-8f0f-4221d9b93709 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:49.914974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b594f9-9c29-4397-bfa7-5d3995f5b9c5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:49.957678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe2026-97bf-4ee0-a446-2ee33ed68a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:49.959349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe2026-97bf-4ee0-a446-2ee33ed68a73 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:49.959960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe2026-97bf-4ee0-a446-2ee33ed68a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:49.961560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe2026-97bf-4ee0-a446-2ee33ed68a73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:49.982051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe2026-97bf-4ee0-a446-2ee33ed68a73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:49.983887 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe2026-97bf-4ee0-a446-2ee33ed68a73 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-bcfe2026-97bf-4ee0-a446-2ee33ed68a73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:19:50.006915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:50.007980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:50.009126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:19:50.009126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:19:50.009126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:19:50.009126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:19:50.009126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:19:50.009126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:19:50.009126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:50.009126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:50.016995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 274/536] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:50.022143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe2026-97bf-4ee0-a446-2ee33ed68a73 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:50.065652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2d3c1-dd2b-4275-8f0f-4221d9b93709 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:50.114630 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:50.156254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc05bc-648c-468b-8f4b-c8ff8050faa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:50.157078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc05bc-648c-468b-8f4b-c8ff8050faa4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:50.157453 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc05bc-648c-468b-8f4b-c8ff8050faa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:50.157822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc05bc-648c-468b-8f4b-c8ff8050faa4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:50.159322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe2026-97bf-4ee0-a446-2ee33ed68a73 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:50.173115 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc05bc-648c-468b-8f4b-c8ff8050faa4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:50.174572 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc05bc-648c-468b-8f4b-c8ff8050faa4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:50.196816 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709b996-01ae-405d-89b7-c87e82d6a2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-a1fc05bc-648c-468b-8f4b-c8ff8050faa4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:19:50.197312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:50.198366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:50.199864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:19:50.199864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:19:50.199864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:19:50.199864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:19:50.199864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:19:50.199864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:19:50.199864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:50.199864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:50.199864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 263/537] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:50.201083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b996-01ae-405d-89b7-c87e82d6a2de None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:50.201573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b996-01ae-405d-89b7-c87e82d6a2de None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:50.202050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b996-01ae-405d-89b7-c87e82d6a2de None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:50.203051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709b996-01ae-405d-89b7-c87e82d6a2de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:50.203553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709b996-01ae-405d-89b7-c87e82d6a2de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:50.204092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc05bc-648c-468b-8f4b-c8ff8050faa4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:50.205476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709b996-01ae-405d-89b7-c87e82d6a2de None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:50.225592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 275/538] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:19:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:50.234496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709b996-01ae-405d-89b7-c87e82d6a2de None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:50.252066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:50.377000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc05bc-648c-468b-8f4b-c8ff8050faa4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:50.386904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709b996-01ae-405d-89b7-c87e82d6a2de None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:50.410403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:50.411372 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cee9e-ba0c-42cb-b95d-e942adbefc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:50.412227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cee9e-ba0c-42cb-b95d-e942adbefc9d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:50.412644 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cee9e-ba0c-42cb-b95d-e942adbefc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:50.413112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cee9e-ba0c-42cb-b95d-e942adbefc9d None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:50.413969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3cee9e-ba0c-42cb-b95d-e942adbefc9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:50.414444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3cee9e-ba0c-42cb-b95d-e942adbefc9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:50.416123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3cee9e-ba0c-42cb-b95d-e942adbefc9d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:50.427016 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00320c-a65b-4cfa-9757-05d408be318a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:50.427773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00320c-a65b-4cfa-9757-05d408be318a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:50.428119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00320c-a65b-4cfa-9757-05d408be318a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:50.428464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00320c-a65b-4cfa-9757-05d408be318a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:50.429266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00320c-a65b-4cfa-9757-05d408be318a None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=407b8483d4f34be19d2c7c7b6013b313) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:50.429644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00320c-a65b-4cfa-9757-05d408be318a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:50.430829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 264/539] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:19:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:50.431289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00320c-a65b-4cfa-9757-05d408be318a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:50.432807 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197002c-a15f-4ef9-b7ba-fe7aeee1e7f8 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:50.433415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197002c-a15f-4ef9-b7ba-fe7aeee1e7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:50.433894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197002c-a15f-4ef9-b7ba-fe7aeee1e7f8 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:50.435632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 265/540] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:50.436560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197002c-a15f-4ef9-b7ba-fe7aeee1e7f8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:50.448554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 276/541] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:50 2022] GET /identity/v3/role_assignments?user.id=407b8483d4f34be19d2c7c7b6013b313&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:50.448554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8226480-c33b-4029-ac0c-685a8487d1eb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:50.448554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8226480-c33b-4029-ac0c-685a8487d1eb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:50.448554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8226480-c33b-4029-ac0c-685a8487d1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:50.468180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f8226480-c33b-4029-ac0c-685a8487d1eb None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:50.468180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f8226480-c33b-4029-ac0c-685a8487d1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:50.552156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 277/542] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:19:50.573623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8226480-c33b-4029-ac0c-685a8487d1eb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:50.620598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:50.664798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197002c-a15f-4ef9-b7ba-fe7aeee1e7f8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:50.695100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8226480-c33b-4029-ac0c-685a8487d1eb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:50.713094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:50.723392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb40e6e-de0a-4fe6-ab57-7595e6bdd244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:50.724301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb40e6e-de0a-4fe6-ab57-7595e6bdd244 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:50.729956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb40e6e-de0a-4fe6-ab57-7595e6bdd244 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:50.729956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb40e6e-de0a-4fe6-ab57-7595e6bdd244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:50.729956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb40e6e-de0a-4fe6-ab57-7595e6bdd244 None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=baffd4f6a8ad4a6e91d8d9c61cc1dce0) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:50.729956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb40e6e-de0a-4fe6-ab57-7595e6bdd244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:50.729956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb40e6e-de0a-4fe6-ab57-7595e6bdd244 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:50.740056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb97982-c7f9-4b44-8318-aad23c702eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:50.740056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb97982-c7f9-4b44-8318-aad23c702eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:50.740056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb97982-c7f9-4b44-8318-aad23c702eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:50.740056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb97982-c7f9-4b44-8318-aad23c702eb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:50.743969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb97982-c7f9-4b44-8318-aad23c702eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:50.747898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 266/543] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:50 2022] GET /identity/v3/role_assignments?user.id=baffd4f6a8ad4a6e91d8d9c61cc1dce0&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-5fb97982-c7f9-4b44-8318-aad23c702eb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:50.749241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:50.750114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:50.750935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:50.755325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 278/544] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:50.756866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb40e6e-de0a-4fe6-ab57-7595e6bdd244 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:50.786144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e23e30-d77b-444e-881d-f598615fa035 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:50.786450 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e23e30-d77b-444e-881d-f598615fa035 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:50.786695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e23e30-d77b-444e-881d-f598615fa035 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:50.788459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 279/545] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:19:50.795133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e3323-119a-4d63-befa-45b1b658ad8f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:50.795422 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e3323-119a-4d63-befa-45b1b658ad8f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:50.795756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e3323-119a-4d63-befa-45b1b658ad8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:50.835863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3e5e3323-119a-4d63-befa-45b1b658ad8f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:50.837139 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3e5e3323-119a-4d63-befa-45b1b658ad8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:50.926885 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 280/546] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:50.966702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e3323-119a-4d63-befa-45b1b658ad8f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:50.994386 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb40e6e-de0a-4fe6-ab57-7595e6bdd244 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:51.041179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2795-8f46-4e92-a6c9-541948453582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:51.043522 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2795-8f46-4e92-a6c9-541948453582 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:51.045894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2795-8f46-4e92-a6c9-541948453582 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:51.046409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2795-8f46-4e92-a6c9-541948453582 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:51.047283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd2795-8f46-4e92-a6c9-541948453582 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:51.049528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd2795-8f46-4e92-a6c9-541948453582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:51.051127 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd2795-8f46-4e92-a6c9-541948453582 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:51.077982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 267/547] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:51.085401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2795-8f46-4e92-a6c9-541948453582 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:51.099533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:51.188343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e3323-119a-4d63-befa-45b1b658ad8f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:51.194221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:51.234808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a3637-e0e1-490b-bc3f-f38a66aab818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:51.235840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3637-e0e1-490b-bc3f-f38a66aab818 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:51.236360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3637-e0e1-490b-bc3f-f38a66aab818 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:51.236830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3637-e0e1-490b-bc3f-f38a66aab818 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:51.249815 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a3637-e0e1-490b-bc3f-f38a66aab818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-212a3637-e0e1-490b-bc3f-f38a66aab818 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:51.254846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:51.255727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:51.256624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:51.268085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 281/548] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:51.269494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a3637-e0e1-490b-bc3f-f38a66aab818 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:51.295024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2795-8f46-4e92-a6c9-541948453582 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:51.361734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dfd6d-cf4d-41bd-8a77-35c0527ded41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:51.362365 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dfd6d-cf4d-41bd-8a77-35c0527ded41 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:51.362640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dfd6d-cf4d-41bd-8a77-35c0527ded41 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:51.362911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dfd6d-cf4d-41bd-8a77-35c0527ded41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:51.380794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dfd6d-cf4d-41bd-8a77-35c0527ded41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:51.382386 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dfd6d-cf4d-41bd-8a77-35c0527ded41 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-e64dfd6d-cf4d-41bd-8a77-35c0527ded41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:51.410642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:51.411747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:51.413235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:51.413235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:51.413235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:51.416116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 268/549] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:51.418160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dfd6d-cf4d-41bd-8a77-35c0527ded41 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:51.478802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:51.494210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a3637-e0e1-490b-bc3f-f38a66aab818 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:51.522376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dfd6d-cf4d-41bd-8a77-35c0527ded41 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:51.559381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966974e-751a-4b29-996f-88e1ef464aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:51.559985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966974e-751a-4b29-996f-88e1ef464aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:51.560292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966974e-751a-4b29-996f-88e1ef464aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:51.560567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966974e-751a-4b29-996f-88e1ef464aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:51.561213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966974e-751a-4b29-996f-88e1ef464aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:51.561519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966974e-751a-4b29-996f-88e1ef464aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:51.563811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966974e-751a-4b29-996f-88e1ef464aa2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:51.582604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48070f-9830-4623-bd44-2371edfcbd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:51.583277 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48070f-9830-4623-bd44-2371edfcbd69 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:51.583560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48070f-9830-4623-bd44-2371edfcbd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:51.583829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48070f-9830-4623-bd44-2371edfcbd69 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:51.584524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48070f-9830-4623-bd44-2371edfcbd69 None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:51.584827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48070f-9830-4623-bd44-2371edfcbd69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:51.586204 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48070f-9830-4623-bd44-2371edfcbd69 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:51.597071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 269/550] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:51.598425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966974e-751a-4b29-996f-88e1ef464aa2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:51.607828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 282/551] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:51.613281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48070f-9830-4623-bd44-2371edfcbd69 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:51.661331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:51.708495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966974e-751a-4b29-996f-88e1ef464aa2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:51.728448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:51.736208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48070f-9830-4623-bd44-2371edfcbd69 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:51.747360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c797a-70f2-4ac6-a240-ecde27eb4624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:51.748485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c797a-70f2-4ac6-a240-ecde27eb4624 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:51.748914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c797a-70f2-4ac6-a240-ecde27eb4624 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:51.749323 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c797a-70f2-4ac6-a240-ecde27eb4624 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:51.757754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c797a-70f2-4ac6-a240-ecde27eb4624 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:51.759296 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c797a-70f2-4ac6-a240-ecde27eb4624 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-1a6c797a-70f2-4ac6-a240-ecde27eb4624 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:19:51.771277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:51.772408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:51.773358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:51.773358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:51.773358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:19:51.773358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:19:51.773358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:19:51.773358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:19:51.773358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:19:51.773358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:19:51.773358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:19:51.773358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:51.776607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 270/552] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:19:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:51.778038 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918b55b-b313-4a58-8b36-f6d60482b10c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:51.778631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918b55b-b313-4a58-8b36-f6d60482b10c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:51.779455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918b55b-b313-4a58-8b36-f6d60482b10c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:51.780998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 271/553] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:51.781718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918b55b-b313-4a58-8b36-f6d60482b10c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:51.783508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a169065-a9dd-486a-ad46-3df91e5f43d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:51.784453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a169065-a9dd-486a-ad46-3df91e5f43d5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:51.784939 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a169065-a9dd-486a-ad46-3df91e5f43d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:51.785403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a169065-a9dd-486a-ad46-3df91e5f43d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:51.800610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a169065-a9dd-486a-ad46-3df91e5f43d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:51.802365 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a169065-a9dd-486a-ad46-3df91e5f43d5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-3a169065-a9dd-486a-ad46-3df91e5f43d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:51.817594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:51.818594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:51.819457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:51.819457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:51.819457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:51.821520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 283/554] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:51.823726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364a5c-2b56-4865-8232-e044a31ee4d8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:51.824250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364a5c-2b56-4865-8232-e044a31ee4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:51.824667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364a5c-2b56-4865-8232-e044a31ee4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:51.856840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cd364a5c-2b56-4865-8232-e044a31ee4d8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:51.858274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cd364a5c-2b56-4865-8232-e044a31ee4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:51.902665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918b55b-b313-4a58-8b36-f6d60482b10c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:51.930027 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 284/555] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:51.930889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd364a5c-2b56-4865-8232-e044a31ee4d8 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:51.940351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e55914-6b07-4dd4-b382-18d35bbf50eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:51.941172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:51.942210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e55914-6b07-4dd4-b382-18d35bbf50eb None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:51.942895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e55914-6b07-4dd4-b382-18d35bbf50eb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:51.943368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e55914-6b07-4dd4-b382-18d35bbf50eb None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:51.944338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e55914-6b07-4dd4-b382-18d35bbf50eb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:51.944842 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e55914-6b07-4dd4-b382-18d35bbf50eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:51.946977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e55914-6b07-4dd4-b382-18d35bbf50eb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:51.966585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 272/556] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 22:19:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:51.967792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e55914-6b07-4dd4-b382-18d35bbf50eb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:52.007197 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:52.040239 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd364a5c-2b56-4865-8232-e044a31ee4d8 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:52.078438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee97b5-f162-4dd0-95e5-0a15984db66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:52.079066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee97b5-f162-4dd0-95e5-0a15984db66f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:52.079360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee97b5-f162-4dd0-95e5-0a15984db66f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:52.079917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee97b5-f162-4dd0-95e5-0a15984db66f None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:52.080416 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee97b5-f162-4dd0-95e5-0a15984db66f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:52.080722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee97b5-f162-4dd0-95e5-0a15984db66f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:52.082104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee97b5-f162-4dd0-95e5-0a15984db66f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:52.083665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e55914-6b07-4dd4-b382-18d35bbf50eb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:52.099367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 285/557] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:52.099367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee97b5-f162-4dd0-95e5-0a15984db66f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:52.126652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677a823-9814-47a9-a420-ac61cfb04f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:52.127948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677a823-9814-47a9-a420-ac61cfb04f1c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:52.128656 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677a823-9814-47a9-a420-ac61cfb04f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:52.129075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677a823-9814-47a9-a420-ac61cfb04f1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:52.143318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677a823-9814-47a9-a420-ac61cfb04f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-0677a823-9814-47a9-a420-ac61cfb04f1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:52.148707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:52.150262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:52.151231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:52.152443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 273/558] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:52.153428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677a823-9814-47a9-a420-ac61cfb04f1c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:52.186006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:52.232456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee97b5-f162-4dd0-95e5-0a15984db66f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:52.254665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:52.261861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677a823-9814-47a9-a420-ac61cfb04f1c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:52.277597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dfbe3-089d-4beb-b62d-8e670dd6a4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:52.278351 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dfbe3-089d-4beb-b62d-8e670dd6a4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:52.278739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dfbe3-089d-4beb-b62d-8e670dd6a4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:52.279015 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dfbe3-089d-4beb-b62d-8e670dd6a4e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:52.299479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310e04a-afe7-4812-a26e-92c2ea62bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:52.300471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310e04a-afe7-4812-a26e-92c2ea62bdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:52.300953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310e04a-afe7-4812-a26e-92c2ea62bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:52.301424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310e04a-afe7-4812-a26e-92c2ea62bdd3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:52.309829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310e04a-afe7-4812-a26e-92c2ea62bdd3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:52.311612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310e04a-afe7-4812-a26e-92c2ea62bdd3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f310e04a-afe7-4812-a26e-92c2ea62bdd3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:19:52.323906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:52.324999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:52.325959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:52.325959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:52.325959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:19:52.325959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:19:52.325959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:19:52.325959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:19:52.325959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:19:52.325959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:19:52.325959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:19:52.325959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:52.328581 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 274/559] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 22:19:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:52.329564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310e04a-afe7-4812-a26e-92c2ea62bdd3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:52.366315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dfbe3-089d-4beb-b62d-8e670dd6a4e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:52.367791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dfbe3-089d-4beb-b62d-8e670dd6a4e5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:52.440633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310e04a-afe7-4812-a26e-92c2ea62bdd3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:52.452331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:52.486153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5bd1e-3d2d-49f7-b666-f0992b47b417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:52.487161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5bd1e-3d2d-49f7-b666-f0992b47b417 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:52.487697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5bd1e-3d2d-49f7-b666-f0992b47b417 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:52.488199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5bd1e-3d2d-49f7-b666-f0992b47b417 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:52.489081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5bd1e-3d2d-49f7-b666-f0992b47b417 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:52.489584 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5bd1e-3d2d-49f7-b666-f0992b47b417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:52.491339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5bd1e-3d2d-49f7-b666-f0992b47b417 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:52.492431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 286/560] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:52.493455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dfbe3-089d-4beb-b62d-8e670dd6a4e5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:52.503374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 275/561] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:52.504627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5bd1e-3d2d-49f7-b666-f0992b47b417 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:52.525707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:52.618724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dfbe3-089d-4beb-b62d-8e670dd6a4e5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:52.619777 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5bd1e-3d2d-49f7-b666-f0992b47b417 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:52.657964 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b2b99-98a1-45cd-a6f0-ffd6eefd1ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:52.658974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b2b99-98a1-45cd-a6f0-ffd6eefd1ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:52.659452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b2b99-98a1-45cd-a6f0-ffd6eefd1ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:52.659921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b2b99-98a1-45cd-a6f0-ffd6eefd1ea2 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:52.661183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b2b99-98a1-45cd-a6f0-ffd6eefd1ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:52.661709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b2b99-98a1-45cd-a6f0-ffd6eefd1ea2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:52.662701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfe536-a251-4d62-bd9a-1fa0c7948d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:52.663330 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfe536-a251-4d62-bd9a-1fa0c7948d16 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:52.663609 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfe536-a251-4d62-bd9a-1fa0c7948d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:52.663881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfe536-a251-4d62-bd9a-1fa0c7948d16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:52.664296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b2b99-98a1-45cd-a6f0-ffd6eefd1ea2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:52.677858 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfe536-a251-4d62-bd9a-1fa0c7948d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:52.679396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfe536-a251-4d62-bd9a-1fa0c7948d16 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:52.685886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 287/562] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 22:19:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:52.686998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b2b99-98a1-45cd-a6f0-ffd6eefd1ea2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-a3dfe536-a251-4d62-bd9a-1fa0c7948d16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:52.696789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:52.697729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:52.699156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:52.699156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:52.699156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:52.699156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 276/563] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:52.699821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73677ce-3258-4aee-ae18-d53ecc400d4b None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:52.699821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73677ce-3258-4aee-ae18-d53ecc400d4b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:52.700142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73677ce-3258-4aee-ae18-d53ecc400d4b None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:52.701677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 277/564] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:52.702623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73677ce-3258-4aee-ae18-d53ecc400d4b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:52.721050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:52.776422 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:52.808166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b2b99-98a1-45cd-a6f0-ffd6eefd1ea2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:52.808840 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73677ce-3258-4aee-ae18-d53ecc400d4b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:52.844126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33207649-5051-4592-9de0-6e0ffbac4a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:52.844764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33207649-5051-4592-9de0-6e0ffbac4a79 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:52.845018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33207649-5051-4592-9de0-6e0ffbac4a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:52.845277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33207649-5051-4592-9de0-6e0ffbac4a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:52.846885 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda92f2-cdcd-42f4-aada-1947d1eff372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:52.847513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda92f2-cdcd-42f4-aada-1947d1eff372 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:52.847790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda92f2-cdcd-42f4-aada-1947d1eff372 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:52.848240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda92f2-cdcd-42f4-aada-1947d1eff372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:52.856838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33207649-5051-4592-9de0-6e0ffbac4a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:52.858264 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33207649-5051-4592-9de0-6e0ffbac4a79 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:52.859106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda92f2-cdcd-42f4-aada-1947d1eff372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:52.860779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda92f2-cdcd-42f4-aada-1947d1eff372 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-33207649-5051-4592-9de0-6e0ffbac4a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:52.873014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:52.873956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:52.879835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:52.879835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:52.879835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:52.879835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 278/565] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:52.879835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33207649-5051-4592-9de0-6e0ffbac4a79 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-1dda92f2-cdcd-42f4-aada-1947d1eff372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:52.880143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:52.880966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:52.882496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:52.882496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:52.882496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:52.882496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 288/566] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:52.882496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258abf-4c13-42f5-a656-ef39c743658d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:52.882496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258abf-4c13-42f5-a656-ef39c743658d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:52.882496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258abf-4c13-42f5-a656-ef39c743658d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:52.907615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1b258abf-4c13-42f5-a656-ef39c743658d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:52.909120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1b258abf-4c13-42f5-a656-ef39c743658d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:52.982707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 289/567] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:52.985178 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33207649-5051-4592-9de0-6e0ffbac4a79 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:52.987298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b258abf-4c13-42f5-a656-ef39c743658d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:52.998781 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:53.009090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:53.020966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c55a48-5646-4c9d-8750-c16a3911cd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.021582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c55a48-5646-4c9d-8750-c16a3911cd84 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.021855 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c55a48-5646-4c9d-8750-c16a3911cd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.022125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c55a48-5646-4c9d-8750-c16a3911cd84 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.022759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c55a48-5646-4c9d-8750-c16a3911cd84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:53.023074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c55a48-5646-4c9d-8750-c16a3911cd84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:53.024434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c55a48-5646-4c9d-8750-c16a3911cd84 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:53.035884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 279/568] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:53.036640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c55a48-5646-4c9d-8750-c16a3911cd84 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.085859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b258abf-4c13-42f5-a656-ef39c743658d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:53.119170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654444fe-3575-4fd2-a1ca-c6da6275e132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.119773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654444fe-3575-4fd2-a1ca-c6da6275e132 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.120597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654444fe-3575-4fd2-a1ca-c6da6275e132 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.120868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654444fe-3575-4fd2-a1ca-c6da6275e132 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.121516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654444fe-3575-4fd2-a1ca-c6da6275e132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:53.121821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654444fe-3575-4fd2-a1ca-c6da6275e132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:53.123187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654444fe-3575-4fd2-a1ca-c6da6275e132 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:53.132735 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c55a48-5646-4c9d-8750-c16a3911cd84 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:53.138601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 290/569] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:53.138601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701e4ec-e871-44fe-89a3-da9530cfe241 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.138601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701e4ec-e871-44fe-89a3-da9530cfe241 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.138601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701e4ec-e871-44fe-89a3-da9530cfe241 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.138601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 291/570] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:53.138601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701e4ec-e871-44fe-89a3-da9530cfe241 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.166459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487ebe0-50ef-4905-bbf7-a62fce787949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.167718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487ebe0-50ef-4905-bbf7-a62fce787949 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.167997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487ebe0-50ef-4905-bbf7-a62fce787949 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.168299 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487ebe0-50ef-4905-bbf7-a62fce787949 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.168933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487ebe0-50ef-4905-bbf7-a62fce787949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:53.169236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487ebe0-50ef-4905-bbf7-a62fce787949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:53.170594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487ebe0-50ef-4905-bbf7-a62fce787949 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:53.181355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 280/571] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:53.182130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487ebe0-50ef-4905-bbf7-a62fce787949 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.205951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:53.228758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:53.240550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701e4ec-e871-44fe-89a3-da9530cfe241 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:53.273449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4444d0b-f8b8-4835-af65-ff7110bdadb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.274068 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4444d0b-f8b8-4835-af65-ff7110bdadb4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.274356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4444d0b-f8b8-4835-af65-ff7110bdadb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.274634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4444d0b-f8b8-4835-af65-ff7110bdadb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.285004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4444d0b-f8b8-4835-af65-ff7110bdadb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:53.286306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487ebe0-50ef-4905-bbf7-a62fce787949 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c4444d0b-f8b8-4835-af65-ff7110bdadb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:53.290774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:53.291702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:53.292921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 292/572] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:53.293707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4444d0b-f8b8-4835-af65-ff7110bdadb4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.321857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c17e0-eb90-4d3b-a990-a0a76cd6e8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.322475 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c17e0-eb90-4d3b-a990-a0a76cd6e8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.322750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c17e0-eb90-4d3b-a990-a0a76cd6e8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.323019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c17e0-eb90-4d3b-a990-a0a76cd6e8c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.329608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c17e0-eb90-4d3b-a990-a0a76cd6e8c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:53.331015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c17e0-eb90-4d3b-a990-a0a76cd6e8c4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-026c17e0-eb90-4d3b-a990-a0a76cd6e8c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:19:53.342403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:53.343465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:53.344383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:19:53.344383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:19:53.344383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:19:53.344383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:19:53.344383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:19:53.344383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:19:53.344383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:53.344383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:53.344383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 281/573] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:53.344885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c508e48-9b7b-43ed-8ae9-a03439d3493b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.345286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c508e48-9b7b-43ed-8ae9-a03439d3493b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.345449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c508e48-9b7b-43ed-8ae9-a03439d3493b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.362859 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7c508e48-9b7b-43ed-8ae9-a03439d3493b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:53.364079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7c508e48-9b7b-43ed-8ae9-a03439d3493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:53.393550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4444d0b-f8b8-4835-af65-ff7110bdadb4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:53.418950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:53.426298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bbe82-96a0-4179-b3a6-facea72ecbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.426919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bbe82-96a0-4179-b3a6-facea72ecbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.427196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bbe82-96a0-4179-b3a6-facea72ecbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.427466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bbe82-96a0-4179-b3a6-facea72ecbf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.429249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 282/574] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:53.430071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c508e48-9b7b-43ed-8ae9-a03439d3493b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.434403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4bbe82-96a0-4179-b3a6-facea72ecbf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:53.435860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4bbe82-96a0-4179-b3a6-facea72ecbf4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-1f4bbe82-96a0-4179-b3a6-facea72ecbf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:19:53.446871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:53.447774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:53.449338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:19:53.449338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:19:53.449338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:19:53.449338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:19:53.449338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:19:53.449338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:19:53.449338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:53.449338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:53.449338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 293/575] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:53.450849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bbe82-96a0-4179-b3a6-facea72ecbf4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.473394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:53.542010 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c508e48-9b7b-43ed-8ae9-a03439d3493b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:53.548381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bbe82-96a0-4179-b3a6-facea72ecbf4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:53.575760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1807e5c-f680-4e48-b06c-9d28745bc325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.576402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1807e5c-f680-4e48-b06c-9d28745bc325 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.576675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1807e5c-f680-4e48-b06c-9d28745bc325 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.576949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1807e5c-f680-4e48-b06c-9d28745bc325 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.581502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db8c2b-1302-4547-8ab7-f2a54cce9920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.582101 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db8c2b-1302-4547-8ab7-f2a54cce9920 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.582379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db8c2b-1302-4547-8ab7-f2a54cce9920 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.582650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db8c2b-1302-4547-8ab7-f2a54cce9920 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.583305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db8c2b-1302-4547-8ab7-f2a54cce9920 None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:53.583607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db8c2b-1302-4547-8ab7-f2a54cce9920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:53.583693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1807e5c-f680-4e48-b06c-9d28745bc325 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:53.585001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db8c2b-1302-4547-8ab7-f2a54cce9920 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:53.585096 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1807e5c-f680-4e48-b06c-9d28745bc325 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:53.593400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 294/576] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:53.594185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db8c2b-1302-4547-8ab7-f2a54cce9920 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d1807e5c-f680-4e48-b06c-9d28745bc325 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:19:53.596830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:53.597751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 283/577] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:53.598724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1807e5c-f680-4e48-b06c-9d28745bc325 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.646820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:53.688401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:53.696857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1807e5c-f680-4e48-b06c-9d28745bc325 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:53.697213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db8c2b-1302-4547-8ab7-f2a54cce9920 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:53.731271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9345d-ce21-4b59-b1fd-3f3fa124ec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.731632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae27307-be23-42dc-bac1-12c7d3ff4cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.731875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9345d-ce21-4b59-b1fd-3f3fa124ec48 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.732187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9345d-ce21-4b59-b1fd-3f3fa124ec48 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.732281 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae27307-be23-42dc-bac1-12c7d3ff4cbf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.732460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9345d-ce21-4b59-b1fd-3f3fa124ec48 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.732549 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae27307-be23-42dc-bac1-12c7d3ff4cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.732803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae27307-be23-42dc-bac1-12c7d3ff4cbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.733139 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9345d-ce21-4b59-b1fd-3f3fa124ec48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:53.733443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9345d-ce21-4b59-b1fd-3f3fa124ec48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:53.734876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9345d-ce21-4b59-b1fd-3f3fa124ec48 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:53.742576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae27307-be23-42dc-bac1-12c7d3ff4cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:53.746808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 295/578] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:53.747584 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9345d-ce21-4b59-b1fd-3f3fa124ec48 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-cae27307-be23-42dc-bac1-12c7d3ff4cbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:53.748662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:53.749527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:53.750351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 284/579] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:53.751041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae27307-be23-42dc-bac1-12c7d3ff4cbf None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.847979 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9345d-ce21-4b59-b1fd-3f3fa124ec48 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:53.849457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae27307-be23-42dc-bac1-12c7d3ff4cbf None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:53.859068 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:53.881607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccb07e-020c-4f57-a5ac-c1de1de4e691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.882199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccb07e-020c-4f57-a5ac-c1de1de4e691 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.882474 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccb07e-020c-4f57-a5ac-c1de1de4e691 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.885861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccb07e-020c-4f57-a5ac-c1de1de4e691 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.885990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718fc1d-596c-46c9-95fd-ada85bed3a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:53.886087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccb07e-020c-4f57-a5ac-c1de1de4e691 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:53.886160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718fc1d-596c-46c9-95fd-ada85bed3a96 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:53.886230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccb07e-020c-4f57-a5ac-c1de1de4e691 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:53.886390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718fc1d-596c-46c9-95fd-ada85bed3a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:53.886390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718fc1d-596c-46c9-95fd-ada85bed3a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:53.886519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccb07e-020c-4f57-a5ac-c1de1de4e691 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:53.894249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718fc1d-596c-46c9-95fd-ada85bed3a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:53.895705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718fc1d-596c-46c9-95fd-ada85bed3a96 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:53.896484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 296/580] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:53.897273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccb07e-020c-4f57-a5ac-c1de1de4e691 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-1718fc1d-596c-46c9-95fd-ada85bed3a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:53.910730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:53.911694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:53.912791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:53.912791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:53.912791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:53.916316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 285/581] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:53.917680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718fc1d-596c-46c9-95fd-ada85bed3a96 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:53.921558 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:53.999255 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccb07e-020c-4f57-a5ac-c1de1de4e691 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:54.019556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718fc1d-596c-46c9-95fd-ada85bed3a96 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:54.033881 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193d3a6-2281-46ec-ae2c-5f58964c9134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:54.034487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193d3a6-2281-46ec-ae2c-5f58964c9134 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:54.034764 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193d3a6-2281-46ec-ae2c-5f58964c9134 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:54.035034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193d3a6-2281-46ec-ae2c-5f58964c9134 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:54.035757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193d3a6-2281-46ec-ae2c-5f58964c9134 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:54.036088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193d3a6-2281-46ec-ae2c-5f58964c9134 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:54.037534 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193d3a6-2281-46ec-ae2c-5f58964c9134 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:54.053371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb16626-68c9-439f-a888-25c16c7ef05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:54.053961 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb16626-68c9-439f-a888-25c16c7ef05a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:54.054235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb16626-68c9-439f-a888-25c16c7ef05a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:54.054501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb16626-68c9-439f-a888-25c16c7ef05a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:54.055212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb16626-68c9-439f-a888-25c16c7ef05a None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=e86991778c2b47c58f10d9d9e3ca5e82) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:54.055639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb16626-68c9-439f-a888-25c16c7ef05a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:54.055743 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 297/582] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:54.056350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193d3a6-2281-46ec-ae2c-5f58964c9134 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:54.056957 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb16626-68c9-439f-a888-25c16c7ef05a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:54.062813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 286/583] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:19:53 2022] GET /identity/v3/role_assignments?user.id=e86991778c2b47c58f10d9d9e3ca5e82&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:54.063582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb16626-68c9-439f-a888-25c16c7ef05a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:54.092959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:54.141816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:54.177708 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb16626-68c9-439f-a888-25c16c7ef05a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:54.178467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193d3a6-2281-46ec-ae2c-5f58964c9134 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:54.217249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdaa0ff-eeba-4675-8da3-67349f88318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:54.217906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaa0ff-eeba-4675-8da3-67349f88318b None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:54.218931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaa0ff-eeba-4675-8da3-67349f88318b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:54.219208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaa0ff-eeba-4675-8da3-67349f88318b None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/8effbd0670ef41f59b46feb9f0997554/roles/900062fd06bc4f00add25eb39325653f` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:54.220090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66847076-2772-469f-92d0-4f24ca219a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:54.220702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66847076-2772-469f-92d0-4f24ca219a6b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:54.220980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66847076-2772-469f-92d0-4f24ca219a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:54.221249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66847076-2772-469f-92d0-4f24ca219a6b None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:54.221907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66847076-2772-469f-92d0-4f24ca219a6b None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:54.222215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66847076-2772-469f-92d0-4f24ca219a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:54.223689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66847076-2772-469f-92d0-4f24ca219a6b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:54.232675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 298/584] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:54.233501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66847076-2772-469f-92d0-4f24ca219a6b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:54.241778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdaa0ff-eeba-4675-8da3-67349f88318b None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=8effbd0670ef41f59b46feb9f0997554, role_id=900062fd06bc4f00add25eb39325653f)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:54.243211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdaa0ff-eeba-4675-8da3-67349f88318b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:54.307868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 287/585] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:19:54 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/8effbd0670ef41f59b46feb9f0997554/roles/900062fd06bc4f00add25eb39325653f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:54.309233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdaa0ff-eeba-4675-8da3-67349f88318b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:54.336364 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:54.348275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66847076-2772-469f-92d0-4f24ca219a6b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:54.388924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e084-1a12-46c6-b1aa-e6147579a169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:54.389537 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e084-1a12-46c6-b1aa-e6147579a169 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:54.389814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e084-1a12-46c6-b1aa-e6147579a169 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:54.390081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e084-1a12-46c6-b1aa-e6147579a169 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:54.390730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8e084-1a12-46c6-b1aa-e6147579a169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:54.391031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8e084-1a12-46c6-b1aa-e6147579a169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:54.392615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8e084-1a12-46c6-b1aa-e6147579a169 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:54.406957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 299/586] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:54.408325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e084-1a12-46c6-b1aa-e6147579a169 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:54.459893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdaa0ff-eeba-4675-8da3-67349f88318b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:54.486941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:54.511494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61ff72-0260-4757-8505-3fe268a5954d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:54.512285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61ff72-0260-4757-8505-3fe268a5954d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:54.512579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61ff72-0260-4757-8505-3fe268a5954d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:54.512831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61ff72-0260-4757-8505-3fe268a5954d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:54.529211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61ff72-0260-4757-8505-3fe268a5954d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:54.529407 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e084-1a12-46c6-b1aa-e6147579a169 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:54.531297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61ff72-0260-4757-8505-3fe268a5954d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-6a61ff72-0260-4757-8505-3fe268a5954d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:54.554060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:54.555090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:54.555958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:54.555958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:54.555958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:54.557765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 288/587] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:54.558584 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61ff72-0260-4757-8505-3fe268a5954d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:54.575374 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84af50-460a-4920-a8dc-443bc20e0351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:54.576081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84af50-460a-4920-a8dc-443bc20e0351 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:54.576369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84af50-460a-4920-a8dc-443bc20e0351 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:54.576639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84af50-460a-4920-a8dc-443bc20e0351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:54.593288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84af50-460a-4920-a8dc-443bc20e0351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:54.595050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84af50-460a-4920-a8dc-443bc20e0351 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-eb84af50-460a-4920-a8dc-443bc20e0351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:54.613464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:54.614462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:54.615335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:54.615335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:54.615335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:54.615335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 300/588] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:54.615994 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84af50-460a-4920-a8dc-443bc20e0351 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:54.619184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:54.680823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61ff72-0260-4757-8505-3fe268a5954d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:54.727677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a14670-3842-4baa-bb8b-059c5be6c056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:54.728509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a14670-3842-4baa-bb8b-059c5be6c056 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:54.728793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a14670-3842-4baa-bb8b-059c5be6c056 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:54.729067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a14670-3842-4baa-bb8b-059c5be6c056 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:54.737936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a14670-3842-4baa-bb8b-059c5be6c056 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:54.739435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a14670-3842-4baa-bb8b-059c5be6c056 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:54.743741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84af50-460a-4920-a8dc-443bc20e0351 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-92a14670-3842-4baa-bb8b-059c5be6c056 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:19:54.751892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:54.752888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:54.753881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:54.753881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:54.753881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:19:54.753881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:19:54.753881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:19:54.753881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:19:54.753881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:19:54.753881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:19:54.753881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 22:19:54.753881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:54.757847 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 289/589] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:54.759339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a14670-3842-4baa-bb8b-059c5be6c056 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:54.785894 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084206ed-f0c6-4524-87c3-cb42292e55b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:54.786887 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084206ed-f0c6-4524-87c3-cb42292e55b6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:54.787405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084206ed-f0c6-4524-87c3-cb42292e55b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:54.787899 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084206ed-f0c6-4524-87c3-cb42292e55b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:54.788791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084206ed-f0c6-4524-87c3-cb42292e55b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:54.789290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084206ed-f0c6-4524-87c3-cb42292e55b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:54.790909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084206ed-f0c6-4524-87c3-cb42292e55b6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:54.794649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:54.805196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 301/590] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:54.806302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084206ed-f0c6-4524-87c3-cb42292e55b6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:54.872803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a14670-3842-4baa-bb8b-059c5be6c056 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:54.878231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:54.910459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d1b7d-7c9f-420f-958f-05510e583fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:54.911064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1b7d-7c9f-420f-958f-05510e583fbe None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:54.911345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1b7d-7c9f-420f-958f-05510e583fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:54.911602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1b7d-7c9f-420f-958f-05510e583fbe None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:54.912321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d1b7d-7c9f-420f-958f-05510e583fbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:54.912781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d1b7d-7c9f-420f-958f-05510e583fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:54.914495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d1b7d-7c9f-420f-958f-05510e583fbe None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:54.923659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084206ed-f0c6-4524-87c3-cb42292e55b6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:54.926827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 290/591] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:54.927624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d1b7d-7c9f-420f-958f-05510e583fbe None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:54.967010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc3673-82a3-4814-a5b7-3e4eaca2869c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:54.967890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc3673-82a3-4814-a5b7-3e4eaca2869c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:54.968341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc3673-82a3-4814-a5b7-3e4eaca2869c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:54.968762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc3673-82a3-4814-a5b7-3e4eaca2869c None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:54.969754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc3673-82a3-4814-a5b7-3e4eaca2869c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:54.970206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc3673-82a3-4814-a5b7-3e4eaca2869c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:54.971911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc3673-82a3-4814-a5b7-3e4eaca2869c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:54.991678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 302/592] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 22:19:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:54.992764 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc3673-82a3-4814-a5b7-3e4eaca2869c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:55.026972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:55.037243 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d1b7d-7c9f-420f-958f-05510e583fbe None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:55.072814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169e59e-f91e-4ce8-80de-9b1266dc90b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:55.073972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169e59e-f91e-4ce8-80de-9b1266dc90b5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:55.074250 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169e59e-f91e-4ce8-80de-9b1266dc90b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:55.074526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169e59e-f91e-4ce8-80de-9b1266dc90b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:55.082122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169e59e-f91e-4ce8-80de-9b1266dc90b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:55.083593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169e59e-f91e-4ce8-80de-9b1266dc90b5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-8169e59e-f91e-4ce8-80de-9b1266dc90b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:19:55.095714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:55.096907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:55.097942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:19:55.097942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:19:55.097942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:19:55.097942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:19:55.097942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:19:55.097942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:19:55.097942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:55.097942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:55.097942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 291/593] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:55.098450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169e59e-f91e-4ce8-80de-9b1266dc90b5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:55.110811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc3673-82a3-4814-a5b7-3e4eaca2869c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:55.134443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:55.151360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657b7d6-9d84-409c-ab76-a47cd6c3e9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:55.152125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b7d6-9d84-409c-ab76-a47cd6c3e9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:55.152411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b7d6-9d84-409c-ab76-a47cd6c3e9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:55.152692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b7d6-9d84-409c-ab76-a47cd6c3e9f0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:55.160395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657b7d6-9d84-409c-ab76-a47cd6c3e9f0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:55.161909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657b7d6-9d84-409c-ab76-a47cd6c3e9f0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f657b7d6-9d84-409c-ab76-a47cd6c3e9f0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:19:55.174592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:55.175705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 303/594] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:55.176822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657b7d6-9d84-409c-ab76-a47cd6c3e9f0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:55.222688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169e59e-f91e-4ce8-80de-9b1266dc90b5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:55.269837 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368fee0-ef20-4c2e-ab69-4c32aa47956d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:55.271144 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368fee0-ef20-4c2e-ab69-4c32aa47956d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:55.271810 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368fee0-ef20-4c2e-ab69-4c32aa47956d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:55.272248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368fee0-ef20-4c2e-ab69-4c32aa47956d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:55.287752 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368fee0-ef20-4c2e-ab69-4c32aa47956d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:55.290048 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368fee0-ef20-4c2e-ab69-4c32aa47956d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:55.293401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657b7d6-9d84-409c-ab76-a47cd6c3e9f0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c368fee0-ef20-4c2e-ab69-4c32aa47956d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:55.310940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:55.312112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:55.313177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:55.313177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:55.313177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:55.315259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 292/595] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:55.316319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368fee0-ef20-4c2e-ab69-4c32aa47956d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:55.320120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:55.334187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d248b-29c6-4852-812c-de5fec77ebcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:55.334812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d248b-29c6-4852-812c-de5fec77ebcb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:55.335218 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d248b-29c6-4852-812c-de5fec77ebcb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:55.335594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d248b-29c6-4852-812c-de5fec77ebcb None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:55.336449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d248b-29c6-4852-812c-de5fec77ebcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:55.336868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d248b-29c6-4852-812c-de5fec77ebcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:55.338395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d248b-29c6-4852-812c-de5fec77ebcb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:55.352235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 304/596] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:19:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:55.353199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d248b-29c6-4852-812c-de5fec77ebcb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:55.400569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:55.450937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368fee0-ef20-4c2e-ab69-4c32aa47956d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:55.466925 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d248b-29c6-4852-812c-de5fec77ebcb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:55.497171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60d3b1-97b2-4633-aaa5-9ad29fa95933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:55.498281 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60d3b1-97b2-4633-aaa5-9ad29fa95933 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:55.499090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60d3b1-97b2-4633-aaa5-9ad29fa95933 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:55.499511 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60d3b1-97b2-4633-aaa5-9ad29fa95933 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:55.500688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60d3b1-97b2-4633-aaa5-9ad29fa95933 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:55.501590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60d3b1-97b2-4633-aaa5-9ad29fa95933 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:55.503545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60d3b1-97b2-4633-aaa5-9ad29fa95933 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:55.513282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adc27e-8e08-49f4-a09a-657d0e9ad250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:55.514172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adc27e-8e08-49f4-a09a-657d0e9ad250 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:55.514657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adc27e-8e08-49f4-a09a-657d0e9ad250 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:55.515358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adc27e-8e08-49f4-a09a-657d0e9ad250 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:55.517400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adc27e-8e08-49f4-a09a-657d0e9ad250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:55.517947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adc27e-8e08-49f4-a09a-657d0e9ad250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:55.519634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adc27e-8e08-49f4-a09a-657d0e9ad250 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:55.526988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 293/597] 10.222.0.30 () {56 vars in 1139 bytes} [Sun Aug 7 22:19:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:55.528059 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60d3b1-97b2-4633-aaa5-9ad29fa95933 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:55.536657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 305/598] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:55.538337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adc27e-8e08-49f4-a09a-657d0e9ad250 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:55.609512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:55.655537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60d3b1-97b2-4633-aaa5-9ad29fa95933 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:55.676332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:55.680295 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adc27e-8e08-49f4-a09a-657d0e9ad250 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:55.694485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbda79e-0a7b-4626-92a4-c220ad126fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:55.695459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbda79e-0a7b-4626-92a4-c220ad126fd7 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:55.696144 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbda79e-0a7b-4626-92a4-c220ad126fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:55.697024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbda79e-0a7b-4626-92a4-c220ad126fd7 None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/cfb2129c006f4ddeba7d128a8d43d5de/roles/900062fd06bc4f00add25eb39325653f` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:55.721213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41b56-3b5d-45f3-a2d7-1ed764c78fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:55.722362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41b56-3b5d-45f3-a2d7-1ed764c78fea None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:55.722829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41b56-3b5d-45f3-a2d7-1ed764c78fea None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:55.723297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41b56-3b5d-45f3-a2d7-1ed764c78fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:55.736498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de41b56-3b5d-45f3-a2d7-1ed764c78fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:55.738084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de41b56-3b5d-45f3-a2d7-1ed764c78fea None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-3de41b56-3b5d-45f3-a2d7-1ed764c78fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:55.755028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:55.756466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:55.757356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:55.757356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:55.757356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:55.757356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 306/599] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:55.757590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41b56-3b5d-45f3-a2d7-1ed764c78fea None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:55.859260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbda79e-0a7b-4626-92a4-c220ad126fd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=cfb2129c006f4ddeba7d128a8d43d5de, role_id=900062fd06bc4f00add25eb39325653f)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:55.860943 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbda79e-0a7b-4626-92a4-c220ad126fd7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:55.879069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41b56-3b5d-45f3-a2d7-1ed764c78fea None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:55.914858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739b82c-890d-4559-aecd-4b2dd7d03a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:55.915549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b82c-890d-4559-aecd-4b2dd7d03a22 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:55.915832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b82c-890d-4559-aecd-4b2dd7d03a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:55.916144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b82c-890d-4559-aecd-4b2dd7d03a22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:55.923485 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739b82c-890d-4559-aecd-4b2dd7d03a22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:55.924989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739b82c-890d-4559-aecd-4b2dd7d03a22 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:55.932558 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 294/600] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:19:55 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/cfb2129c006f4ddeba7d128a8d43d5de/roles/900062fd06bc4f00add25eb39325653f => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:55.933251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbda79e-0a7b-4626-92a4-c220ad126fd7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-9739b82c-890d-4559-aecd-4b2dd7d03a22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:19:55.954304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:55.955273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:55.956201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:19:55.956201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:19:55.956201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:19:55.956201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:19:55.956201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:19:55.956201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:19:55.956201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:55.956201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:55.964154 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 307/601] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:55.965457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739b82c-890d-4559-aecd-4b2dd7d03a22 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:55.989227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:56.099230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739b82c-890d-4559-aecd-4b2dd7d03a22 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:56.143851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0163441-ec48-48df-8c6c-d48a24778d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:56.144836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0163441-ec48-48df-8c6c-d48a24778d69 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.145251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0163441-ec48-48df-8c6c-d48a24778d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.145980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0163441-ec48-48df-8c6c-d48a24778d69 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:56.147097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0163441-ec48-48df-8c6c-d48a24778d69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:56.147542 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0163441-ec48-48df-8c6c-d48a24778d69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:56.149778 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0163441-ec48-48df-8c6c-d48a24778d69 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:56.155757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbda79e-0a7b-4626-92a4-c220ad126fd7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:56.167299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 308/602] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:19:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:56.168757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:56.168880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7d60d-b9d3-4915-9dbe-a2da91614bc6 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.172401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7d60d-b9d3-4915-9dbe-a2da91614bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.172855 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7d60d-b9d3-4915-9dbe-a2da91614bc6 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:56.175651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 309/603] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:56.180522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7d60d-b9d3-4915-9dbe-a2da91614bc6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:56.207841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb42ee8-d181-4366-be09-32431f25f689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:56.208739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb42ee8-d181-4366-be09-32431f25f689 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.209249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb42ee8-d181-4366-be09-32431f25f689 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.209529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb42ee8-d181-4366-be09-32431f25f689 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:56.220669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb42ee8-d181-4366-be09-32431f25f689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:56.220669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb42ee8-d181-4366-be09-32431f25f689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:56.220669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb42ee8-d181-4366-be09-32431f25f689 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:56.224758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 295/604] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:56.226066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8168a85-f68f-4f1d-ae56-92f8d493a31b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.226371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8168a85-f68f-4f1d-ae56-92f8d493a31b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.226678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8168a85-f68f-4f1d-ae56-92f8d493a31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:56.247738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f8168a85-f68f-4f1d-ae56-92f8d493a31b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:56.248996 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f8168a85-f68f-4f1d-ae56-92f8d493a31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:56.279283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:56.323610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 296/605] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:56.324613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7d60d-b9d3-4915-9dbe-a2da91614bc6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:56.326841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8168a85-f68f-4f1d-ae56-92f8d493a31b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:56.377058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44d62e-f5a0-45f7-8aea-4832d05fce31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:56.377837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44d62e-f5a0-45f7-8aea-4832d05fce31 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.378226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44d62e-f5a0-45f7-8aea-4832d05fce31 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.378609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44d62e-f5a0-45f7-8aea-4832d05fce31 None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/407b8483d4f34be19d2c7c7b6013b313/roles/900062fd06bc4f00add25eb39325653f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:56.395524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44d62e-f5a0-45f7-8aea-4832d05fce31 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=407b8483d4f34be19d2c7c7b6013b313, role_id=900062fd06bc4f00add25eb39325653f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:56.397184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44d62e-f5a0-45f7-8aea-4832d05fce31 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:56.446585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:56.449838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8168a85-f68f-4f1d-ae56-92f8d493a31b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:56.483480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6149f1-bfea-4d66-bb39-94a7f150dff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:56.484145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6149f1-bfea-4d66-bb39-94a7f150dff1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.484426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6149f1-bfea-4d66-bb39-94a7f150dff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.484699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6149f1-bfea-4d66-bb39-94a7f150dff1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:56.491210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6149f1-bfea-4d66-bb39-94a7f150dff1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:56.492681 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6149f1-bfea-4d66-bb39-94a7f150dff1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-2e6149f1-bfea-4d66-bb39-94a7f150dff1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:19:56.504472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:56.505424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:56.506493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:19:56.506493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:19:56.506493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:19:56.506493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:19:56.506493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:19:56.506493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:19:56.506493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:56.506493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:56.506493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 297/606] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:56.507063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6149f1-bfea-4d66-bb39-94a7f150dff1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:56.515052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 310/607] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:19:56 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/407b8483d4f34be19d2c7c7b6013b313/roles/900062fd06bc4f00add25eb39325653f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:56.517077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547a938-17fd-4243-b56a-0d573cc70a11 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.517478 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547a938-17fd-4243-b56a-0d573cc70a11 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.518282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547a938-17fd-4243-b56a-0d573cc70a11 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:56.520926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 311/608] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:56.522631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547a938-17fd-4243-b56a-0d573cc70a11 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:56.649382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6149f1-bfea-4d66-bb39-94a7f150dff1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:56.673012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547a938-17fd-4243-b56a-0d573cc70a11 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:56.694071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ba60a-c00a-4249-99e5-8ea548b71d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:56.694702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba60a-c00a-4249-99e5-8ea548b71d29 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.694978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba60a-c00a-4249-99e5-8ea548b71d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.695318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba60a-c00a-4249-99e5-8ea548b71d29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:56.706664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ba60a-c00a-4249-99e5-8ea548b71d29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-005ba60a-c00a-4249-99e5-8ea548b71d29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:56.711464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:56.712493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:56.713408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 298/609] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:56.714166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8d93-613f-48a1-ae1a-7eac3ccf2515 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.716449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8d93-613f-48a1-ae1a-7eac3ccf2515 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.716449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8d93-613f-48a1-ae1a-7eac3ccf2515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:56.736400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:56.745893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-360e8d93-613f-48a1-ae1a-7eac3ccf2515 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:56.747328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-360e8d93-613f-48a1-ae1a-7eac3ccf2515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:56.750181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa5202-9242-49af-bdb4-69dcd0a4d0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:56.751074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa5202-9242-49af-bdb4-69dcd0a4d0ba None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.751434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa5202-9242-49af-bdb4-69dcd0a4d0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.751854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa5202-9242-49af-bdb4-69dcd0a4d0ba None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:56.752718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa5202-9242-49af-bdb4-69dcd0a4d0ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:56.753126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa5202-9242-49af-bdb4-69dcd0a4d0ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:56.754654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa5202-9242-49af-bdb4-69dcd0a4d0ba None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:56.770580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 312/610] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:19:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:56.771574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa5202-9242-49af-bdb4-69dcd0a4d0ba None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:56.784398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:56.827145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 299/611] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:56.828248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e8d93-613f-48a1-ae1a-7eac3ccf2515 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:56.914435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa5202-9242-49af-bdb4-69dcd0a4d0ba None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:56.939223 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e8d93-613f-48a1-ae1a-7eac3ccf2515 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:56.958987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c281173-d8b2-438e-9a9b-07462f8efa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:56.960101 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281173-d8b2-438e-9a9b-07462f8efa71 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.960387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281173-d8b2-438e-9a9b-07462f8efa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.960666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281173-d8b2-438e-9a9b-07462f8efa71 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:56.961622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c281173-d8b2-438e-9a9b-07462f8efa71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:56.961929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c281173-d8b2-438e-9a9b-07462f8efa71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:56.964297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c281173-d8b2-438e-9a9b-07462f8efa71 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:56.974452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab8643-a967-4a29-8a49-7aed05e2452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:56.974901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 313/612] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:56.975322 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab8643-a967-4a29-8a49-7aed05e2452a None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:56.975696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab8643-a967-4a29-8a49-7aed05e2452a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:56.975830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c281173-d8b2-438e-9a9b-07462f8efa71 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:56.976171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab8643-a967-4a29-8a49-7aed05e2452a None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/baffd4f6a8ad4a6e91d8d9c61cc1dce0/roles/900062fd06bc4f00add25eb39325653f` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:57.000318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab8643-a967-4a29-8a49-7aed05e2452a None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=baffd4f6a8ad4a6e91d8d9c61cc1dce0, role_id=900062fd06bc4f00add25eb39325653f)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:57.002085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab8643-a967-4a29-8a49-7aed05e2452a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:57.025039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:57.053317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:57.060082 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 300/613] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:19:56 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/baffd4f6a8ad4a6e91d8d9c61cc1dce0/roles/900062fd06bc4f00add25eb39325653f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:57.061246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab8643-a967-4a29-8a49-7aed05e2452a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:57.121857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c281173-d8b2-438e-9a9b-07462f8efa71 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:57.201745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6272f-25fb-4282-87c7-f4d89e0c11af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:57.203015 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6272f-25fb-4282-87c7-f4d89e0c11af None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:57.203604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6272f-25fb-4282-87c7-f4d89e0c11af None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:57.205292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6272f-25fb-4282-87c7-f4d89e0c11af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:57.230934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6272f-25fb-4282-87c7-f4d89e0c11af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-55e6272f-25fb-4282-87c7-f4d89e0c11af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:57.236714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:57.237745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:57.238605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:57.244370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 314/614] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:57.251890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6272f-25fb-4282-87c7-f4d89e0c11af None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:57.283137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab8643-a967-4a29-8a49-7aed05e2452a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:57.336340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0616d-054a-4916-bb87-f4f9c03f059c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:57.336340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0616d-054a-4916-bb87-f4f9c03f059c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:57.336340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0616d-054a-4916-bb87-f4f9c03f059c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:57.336340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0616d-054a-4916-bb87-f4f9c03f059c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:57.356293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0616d-054a-4916-bb87-f4f9c03f059c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:57.357836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0616d-054a-4916-bb87-f4f9c03f059c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-98c0616d-054a-4916-bb87-f4f9c03f059c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:57.373490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:57.374654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:57.375762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:57.375762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:57.375762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:57.376501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 301/615] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:57.381435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0616d-054a-4916-bb87-f4f9c03f059c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:57.419892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6272f-25fb-4282-87c7-f4d89e0c11af None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:57.423930 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:57.467696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:57.504743 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f13ba6-be15-4b28-ac02-d0caacde32b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:57.505448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f13ba6-be15-4b28-ac02-d0caacde32b7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:57.505796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f13ba6-be15-4b28-ac02-d0caacde32b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:57.506135 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f13ba6-be15-4b28-ac02-d0caacde32b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:57.506859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f13ba6-be15-4b28-ac02-d0caacde32b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:57.507301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f13ba6-be15-4b28-ac02-d0caacde32b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:57.508693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f13ba6-be15-4b28-ac02-d0caacde32b7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:57.528609 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0616d-054a-4916-bb87-f4f9c03f059c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:57.530528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 315/616] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:57.535903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f13ba6-be15-4b28-ac02-d0caacde32b7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:57.567835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50ad60-aaa7-4e45-bbaf-c750258b866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:57.568781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50ad60-aaa7-4e45-bbaf-c750258b866f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:57.569242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50ad60-aaa7-4e45-bbaf-c750258b866f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:57.570030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50ad60-aaa7-4e45-bbaf-c750258b866f None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:57.571020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50ad60-aaa7-4e45-bbaf-c750258b866f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:57.571653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50ad60-aaa7-4e45-bbaf-c750258b866f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:57.573613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50ad60-aaa7-4e45-bbaf-c750258b866f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:57.587928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 302/617] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:57.594848 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50ad60-aaa7-4e45-bbaf-c750258b866f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:57.711610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50ad60-aaa7-4e45-bbaf-c750258b866f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:57.727693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:57.752423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba7079-c356-477e-8daa-e725187b30a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:57.753298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba7079-c356-477e-8daa-e725187b30a9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:57.753963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba7079-c356-477e-8daa-e725187b30a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:57.754527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba7079-c356-477e-8daa-e725187b30a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:57.764430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba7079-c356-477e-8daa-e725187b30a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:57.766191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba7079-c356-477e-8daa-e725187b30a9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f5ba7079-c356-477e-8daa-e725187b30a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:19:57.780387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:57.781394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:57.782328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:57.782328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:57.782328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:19:57.782328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:19:57.782328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:19:57.782328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:19:57.782328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:19:57.782328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:19:57.782328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:19:57.782328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:57.784949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 303/618] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:19:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:57.791832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba7079-c356-477e-8daa-e725187b30a9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:57.837959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f13ba6-be15-4b28-ac02-d0caacde32b7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:57.924215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba7079-c356-477e-8daa-e725187b30a9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:57.962233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b67cd-29b2-4f11-9537-7b99f58c2fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:57.963053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b67cd-29b2-4f11-9537-7b99f58c2fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:57.963490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b67cd-29b2-4f11-9537-7b99f58c2fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:57.963877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b67cd-29b2-4f11-9537-7b99f58c2fe2 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:57.964899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b67cd-29b2-4f11-9537-7b99f58c2fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:57.965321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b67cd-29b2-4f11-9537-7b99f58c2fe2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:57.966843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b67cd-29b2-4f11-9537-7b99f58c2fe2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:57.987049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 304/619] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 22:19:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:57.992632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b67cd-29b2-4f11-9537-7b99f58c2fe2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:58.000733 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed302c50-4cf3-454a-b768-1029919c6e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:58.001390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed302c50-4cf3-454a-b768-1029919c6e97 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.001672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed302c50-4cf3-454a-b768-1029919c6e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.001952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed302c50-4cf3-454a-b768-1029919c6e97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.026591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:58.057211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed302c50-4cf3-454a-b768-1029919c6e97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:58.058709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed302c50-4cf3-454a-b768-1029919c6e97 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ed302c50-4cf3-454a-b768-1029919c6e97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:58.090963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:58.091974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:58.092923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:58.092923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:58.092923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:58.096150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 316/620] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 560 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:19:58.101314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed302c50-4cf3-454a-b768-1029919c6e97 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:58.139577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:58.142534 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b67cd-29b2-4f11-9537-7b99f58c2fe2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:58.211027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ef598-a90b-4865-93ef-afdaaa9abc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:58.212094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ef598-a90b-4865-93ef-afdaaa9abc4e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.212658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ef598-a90b-4865-93ef-afdaaa9abc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.213273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ef598-a90b-4865-93ef-afdaaa9abc4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.234659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ef598-a90b-4865-93ef-afdaaa9abc4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:58.236484 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ef598-a90b-4865-93ef-afdaaa9abc4e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-4e1ef598-a90b-4865-93ef-afdaaa9abc4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:58.258233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:58.259195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:58.260109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:58.260109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:58.260109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:58.260109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 305/621] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:58.266628 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ef598-a90b-4865-93ef-afdaaa9abc4e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:58.285378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed302c50-4cf3-454a-b768-1029919c6e97 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:58.328202 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808a01e-f84f-466d-ac26-4ed351eaa36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:58.328955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a01e-f84f-466d-ac26-4ed351eaa36d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.329246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a01e-f84f-466d-ac26-4ed351eaa36d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.329635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a01e-f84f-466d-ac26-4ed351eaa36d None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.330286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808a01e-f84f-466d-ac26-4ed351eaa36d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:58.330593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808a01e-f84f-466d-ac26-4ed351eaa36d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:58.332159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808a01e-f84f-466d-ac26-4ed351eaa36d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:58.349805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 317/622] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:58.351366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c213976-3e42-4372-b1cb-7c89da652c58 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.351658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c213976-3e42-4372-b1cb-7c89da652c58 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.351935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c213976-3e42-4372-b1cb-7c89da652c58 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.358443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 318/623] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:58.359301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c213976-3e42-4372-b1cb-7c89da652c58 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:58.381391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:58.406602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ef598-a90b-4865-93ef-afdaaa9abc4e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:58.453484 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21eddfc-767e-49b7-9ea6-bb4142682c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:58.455305 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21eddfc-767e-49b7-9ea6-bb4142682c76 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.455735 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21eddfc-767e-49b7-9ea6-bb4142682c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.456183 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21eddfc-767e-49b7-9ea6-bb4142682c76 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.456971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21eddfc-767e-49b7-9ea6-bb4142682c76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:58.457419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21eddfc-767e-49b7-9ea6-bb4142682c76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:58.458948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21eddfc-767e-49b7-9ea6-bb4142682c76 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:58.465128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:58.475962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 306/624] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:19:58.477519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4153b9-de31-437c-abdd-e31f5a6f9e8c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.479195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4153b9-de31-437c-abdd-e31f5a6f9e8c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.479195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4153b9-de31-437c-abdd-e31f5a6f9e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.496916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c213976-3e42-4372-b1cb-7c89da652c58 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:58.502681 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2e4153b9-de31-437c-abdd-e31f5a6f9e8c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:58.504054 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2e4153b9-de31-437c-abdd-e31f5a6f9e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:58.541224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad152d-0167-4747-b005-8b477befea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:58.542105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad152d-0167-4747-b005-8b477befea37 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.542489 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad152d-0167-4747-b005-8b477befea37 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.543237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad152d-0167-4747-b005-8b477befea37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.556773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad152d-0167-4747-b005-8b477befea37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:58.559552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad152d-0167-4747-b005-8b477befea37 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-a2ad152d-0167-4747-b005-8b477befea37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:19:58.572779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:58.573827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:58.574989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:58.574989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:58.574989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:19:58.574989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:19:58.574989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:19:58.574989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:19:58.574989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:19:58.574989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:19:58.574989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:19:58.574989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:58.576832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 319/625] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 22:19:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:58.577949 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad152d-0167-4747-b005-8b477befea37 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:58.592008 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 307/626] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:58.593120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4153b9-de31-437c-abdd-e31f5a6f9e8c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:58.683433 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:58.691255 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad152d-0167-4747-b005-8b477befea37 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:58.714997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4153b9-de31-437c-abdd-e31f5a6f9e8c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:58.727714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8dcc4-60f9-489a-878a-67bcc7696df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:58.728487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8dcc4-60f9-489a-878a-67bcc7696df4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.728766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8dcc4-60f9-489a-878a-67bcc7696df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.729100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8dcc4-60f9-489a-878a-67bcc7696df4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.737956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8dcc4-60f9-489a-878a-67bcc7696df4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:58.739866 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8dcc4-60f9-489a-878a-67bcc7696df4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:58.742073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:58.750268 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbcf02c-4ec3-4cef-b127-4eea2a102c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:58.759425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcf02c-4ec3-4cef-b127-4eea2a102c71 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.759425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcf02c-4ec3-4cef-b127-4eea2a102c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.759425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcf02c-4ec3-4cef-b127-4eea2a102c71 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.759425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcf02c-4ec3-4cef-b127-4eea2a102c71 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:58.759425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcf02c-4ec3-4cef-b127-4eea2a102c71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ecd8dcc4-60f9-489a-878a-67bcc7696df4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:19:58.759838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:58.761674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:58.762767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:19:58.762767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:19:58.762767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:19:58.762767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:19:58.762767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:19:58.762767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:19:58.762767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:19:58.762767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:58.763375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcf02c-4ec3-4cef-b127-4eea2a102c71 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:58.763460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 320/627] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:19:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:58.763460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8dcc4-60f9-489a-878a-67bcc7696df4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:58.778850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 308/628] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 22:19:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:58.778850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a70ed-35f2-47e9-abaf-1c7c6d05df7c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.778850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a70ed-35f2-47e9-abaf-1c7c6d05df7c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.778850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a70ed-35f2-47e9-abaf-1c7c6d05df7c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.778850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 309/629] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:58.778850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a70ed-35f2-47e9-abaf-1c7c6d05df7c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:58.878536 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8dcc4-60f9-489a-878a-67bcc7696df4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:58.884397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a70ed-35f2-47e9-abaf-1c7c6d05df7c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:58.914910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dbbfc-7b86-4936-920f-cb31fae13ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:58.915539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dbbfc-7b86-4936-920f-cb31fae13ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.915818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dbbfc-7b86-4936-920f-cb31fae13ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.916051 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:58.916181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dbbfc-7b86-4936-920f-cb31fae13ef4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.921843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0bc84-db47-4f2d-9cfb-25ca1c4b334c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:58.929583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0bc84-db47-4f2d-9cfb-25ca1c4b334c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.929583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0bc84-db47-4f2d-9cfb-25ca1c4b334c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.929583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0bc84-db47-4f2d-9cfb-25ca1c4b334c None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.929583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0bc84-db47-4f2d-9cfb-25ca1c4b334c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:58.929583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0bc84-db47-4f2d-9cfb-25ca1c4b334c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:58.929583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0bc84-db47-4f2d-9cfb-25ca1c4b334c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:58.930016 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dbbfc-7b86-4936-920f-cb31fae13ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-673dbbfc-7b86-4936-920f-cb31fae13ef4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:58.933057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:58.933978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:58.934856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 321/630] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:58.936206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dbbfc-7b86-4936-920f-cb31fae13ef4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:58.942831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 310/631] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:19:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:58.948793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b88909-8d97-4970-90dc-d68557ad8167 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:58.948793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b88909-8d97-4970-90dc-d68557ad8167 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:58.948793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b88909-8d97-4970-90dc-d68557ad8167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:58.964655 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-38b88909-8d97-4970-90dc-d68557ad8167 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:58.965854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-38b88909-8d97-4970-90dc-d68557ad8167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:59.008902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:59.036389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 311/632] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:59.037425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b88909-8d97-4970-90dc-d68557ad8167 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:59.068914 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dbbfc-7b86-4936-920f-cb31fae13ef4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:59.104968 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511859c-e01e-4ffe-955d-4f990391ba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:59.105577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511859c-e01e-4ffe-955d-4f990391ba65 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:59.105881 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511859c-e01e-4ffe-955d-4f990391ba65 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:59.106178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511859c-e01e-4ffe-955d-4f990391ba65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:59.147647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b88909-8d97-4970-90dc-d68557ad8167 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:59.170116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:59.176839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4511859c-e01e-4ffe-955d-4f990391ba65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:59.178304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4511859c-e01e-4ffe-955d-4f990391ba65 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:59.184891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85362110-9fd5-4200-b4d7-38e336934546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:59.185506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85362110-9fd5-4200-b4d7-38e336934546 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:59.185786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85362110-9fd5-4200-b4d7-38e336934546 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:59.186061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85362110-9fd5-4200-b4d7-38e336934546 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:59.186802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85362110-9fd5-4200-b4d7-38e336934546 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:59.187112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85362110-9fd5-4200-b4d7-38e336934546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:59.188518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85362110-9fd5-4200-b4d7-38e336934546 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:59.202129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 312/633] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:59.202129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85362110-9fd5-4200-b4d7-38e336934546 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:59.248349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:59.295969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 322/634] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:19:59.295969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511859c-e01e-4ffe-955d-4f990391ba65 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:59.317505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85362110-9fd5-4200-b4d7-38e336934546 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:59.350486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64197d-2128-4bce-a7fa-93f60b259f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:59.351120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64197d-2128-4bce-a7fa-93f60b259f7e None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:59.351396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64197d-2128-4bce-a7fa-93f60b259f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:59.351675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64197d-2128-4bce-a7fa-93f60b259f7e None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/e86991778c2b47c58f10d9d9e3ca5e82/roles/900062fd06bc4f00add25eb39325653f` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:59.368766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64197d-2128-4bce-a7fa-93f60b259f7e None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=e86991778c2b47c58f10d9d9e3ca5e82, role_id=900062fd06bc4f00add25eb39325653f)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:59.370467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64197d-2128-4bce-a7fa-93f60b259f7e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:59.398323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511859c-e01e-4ffe-955d-4f990391ba65 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:59.431594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26d073-0c77-4d2b-91ab-f7c86231bc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:59.432373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26d073-0c77-4d2b-91ab-f7c86231bc11 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:59.432686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 313/635] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:19:59 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/e86991778c2b47c58f10d9d9e3ca5e82/roles/900062fd06bc4f00add25eb39325653f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:19:59.432801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26d073-0c77-4d2b-91ab-f7c86231bc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:59.432967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26d073-0c77-4d2b-91ab-f7c86231bc11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:59.433528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64197d-2128-4bce-a7fa-93f60b259f7e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:59.444884 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26d073-0c77-4d2b-91ab-f7c86231bc11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-4c26d073-0c77-4d2b-91ab-f7c86231bc11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:19:59.450492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:59.457938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 323/636] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:19:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:59.458790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26d073-0c77-4d2b-91ab-f7c86231bc11 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:59.478828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:59.479856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:59.554430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64197d-2128-4bce-a7fa-93f60b259f7e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:59.574820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26d073-0c77-4d2b-91ab-f7c86231bc11 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:59.596755 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0fbac-8e31-45bd-9e25-b2aa0e630d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:59.597211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0fbac-8e31-45bd-9e25-b2aa0e630d56 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:59.597782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0fbac-8e31-45bd-9e25-b2aa0e630d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:59.598812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0fbac-8e31-45bd-9e25-b2aa0e630d56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:59.610030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0fbac-8e31-45bd-9e25-b2aa0e630d56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:59.612225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0fbac-8e31-45bd-9e25-b2aa0e630d56 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:59.615766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea168a5c-0b03-4d05-aa5f-596942419ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:59.616437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea168a5c-0b03-4d05-aa5f-596942419ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:59.616722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea168a5c-0b03-4d05-aa5f-596942419ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:59.616996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea168a5c-0b03-4d05-aa5f-596942419ea6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:59.627919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea168a5c-0b03-4d05-aa5f-596942419ea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:59.629464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea168a5c-0b03-4d05-aa5f-596942419ea6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-34a0fbac-8e31-45bd-9e25-b2aa0e630d56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:59.629565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:59.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:59.631335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:59.631335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:59.631335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:19:59.631335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 314/637] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:59.632711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4cf93-9e48-4a0d-9a1a-5dafc5d86a86 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:59.632992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4cf93-9e48-4a0d-9a1a-5dafc5d86a86 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:59.633266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4cf93-9e48-4a0d-9a1a-5dafc5d86a86 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:59.639239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 315/638] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:19:59.639239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4cf93-9e48-4a0d-9a1a-5dafc5d86a86 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ea168a5c-0b03-4d05-aa5f-596942419ea6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:19:59.645945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:19:59.646965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:19:59.647842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:19:59.647842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:19:59.647842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:19:59.647842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 324/639] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:19:59.647842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea168a5c-0b03-4d05-aa5f-596942419ea6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:59.723072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:59.731907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:59.747427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4cf93-9e48-4a0d-9a1a-5dafc5d86a86 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:59.756885 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea168a5c-0b03-4d05-aa5f-596942419ea6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:59.781724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390d524-6d3e-4a57-ace9-395ca4986fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:59.782501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390d524-6d3e-4a57-ace9-395ca4986fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:59.783285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390d524-6d3e-4a57-ace9-395ca4986fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:59.783552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390d524-6d3e-4a57-ace9-395ca4986fe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:59.784367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390d524-6d3e-4a57-ace9-395ca4986fe2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:59.784671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390d524-6d3e-4a57-ace9-395ca4986fe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:59.786134 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390d524-6d3e-4a57-ace9-395ca4986fe2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:59.794298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3819c-09cc-42a3-9f1e-02961f34a0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:59.794906 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3819c-09cc-42a3-9f1e-02961f34a0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:59.795179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3819c-09cc-42a3-9f1e-02961f34a0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:59.795279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 316/640] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:19:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:59.795458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3819c-09cc-42a3-9f1e-02961f34a0a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:59.799460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3819c-09cc-42a3-9f1e-02961f34a0a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:59.799460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3819c-09cc-42a3-9f1e-02961f34a0a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:59.799709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26f5e0-1031-43fc-80d4-1d857938ea76 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:59.799709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26f5e0-1031-43fc-80d4-1d857938ea76 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:59.799709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26f5e0-1031-43fc-80d4-1d857938ea76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:59.799936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3819c-09cc-42a3-9f1e-02961f34a0a2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:59.813130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 325/641] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:59.813408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3819c-09cc-42a3-9f1e-02961f34a0a2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:59.819854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-af26f5e0-1031-43fc-80d4-1d857938ea76 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:19:59.821087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-af26f5e0-1031-43fc-80d4-1d857938ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:19:59.893289 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 317/642] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:19:59.894313 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26f5e0-1031-43fc-80d4-1d857938ea76 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:59.928198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3819c-09cc-42a3-9f1e-02961f34a0a2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:19:59.962363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:19:59.968944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c12873-fffe-4bb8-961e-db0d67f86f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:19:59.969551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12873-fffe-4bb8-961e-db0d67f86f6b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:19:59.969826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12873-fffe-4bb8-961e-db0d67f86f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:19:59.975191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12873-fffe-4bb8-961e-db0d67f86f6b None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:19:59.975191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c12873-fffe-4bb8-961e-db0d67f86f6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:19:59.975191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c12873-fffe-4bb8-961e-db0d67f86f6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:19:59.975191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c12873-fffe-4bb8-961e-db0d67f86f6b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:19:59.988339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 326/643] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:19:59.988339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c12873-fffe-4bb8-961e-db0d67f86f6b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:19:59.991179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:00.014964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26f5e0-1031-43fc-80d4-1d857938ea76 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:00.050167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a6c2f-cb7a-4cb1-94e8-38bc7d6c0571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:00.050887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a6c2f-cb7a-4cb1-94e8-38bc7d6c0571 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:00.051225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a6c2f-cb7a-4cb1-94e8-38bc7d6c0571 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:00.051561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a6c2f-cb7a-4cb1-94e8-38bc7d6c0571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:00.062901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a6c2f-cb7a-4cb1-94e8-38bc7d6c0571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:00.064361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a6c2f-cb7a-4cb1-94e8-38bc7d6c0571 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-586a6c2f-cb7a-4cb1-94e8-38bc7d6c0571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:00.083474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:00.084528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:00.085423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:00.085423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:00.085423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:00.089458 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 318/644] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:00.090555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a6c2f-cb7a-4cb1-94e8-38bc7d6c0571 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:00.097509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c12873-fffe-4bb8-961e-db0d67f86f6b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:00.132660 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4006f41-8bd2-4254-8afb-0b390beb04a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:00.133625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4006f41-8bd2-4254-8afb-0b390beb04a3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:00.133625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4006f41-8bd2-4254-8afb-0b390beb04a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:00.133899 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4006f41-8bd2-4254-8afb-0b390beb04a3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:00.142347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4006f41-8bd2-4254-8afb-0b390beb04a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:00.143841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4006f41-8bd2-4254-8afb-0b390beb04a3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f4006f41-8bd2-4254-8afb-0b390beb04a3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:00.155328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:00.156301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:00.157412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:00.157412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:00.157412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:00.157412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:00.157412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:00.157412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:00.157412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:00.157412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:00.157412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 22:20:00.157412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:00.157412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 327/645] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:19:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:00.158024 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4006f41-8bd2-4254-8afb-0b390beb04a3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:00.202462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a6c2f-cb7a-4cb1-94e8-38bc7d6c0571 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:00.215355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:00.242412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:00.242412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01f760-9007-4a35-9d8d-2360dff7f5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:00.242412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01f760-9007-4a35-9d8d-2360dff7f5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:00.242412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01f760-9007-4a35-9d8d-2360dff7f5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:00.243029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01f760-9007-4a35-9d8d-2360dff7f5c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:00.252076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01f760-9007-4a35-9d8d-2360dff7f5c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-fc01f760-9007-4a35-9d8d-2360dff7f5c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:00.256846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:00.257767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:20:00.258607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:00.263774 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 319/646] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:20:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:00.264487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01f760-9007-4a35-9d8d-2360dff7f5c5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:00.275290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4006f41-8bd2-4254-8afb-0b390beb04a3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:00.311093 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b52fe-e25b-4c97-9c41-051225c0d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:00.311714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b52fe-e25b-4c97-9c41-051225c0d0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:00.311994 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b52fe-e25b-4c97-9c41-051225c0d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:00.312298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b52fe-e25b-4c97-9c41-051225c0d0d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:00.319081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b52fe-e25b-4c97-9c41-051225c0d0d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:00.320580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b52fe-e25b-4c97-9c41-051225c0d0d5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ef6b52fe-e25b-4c97-9c41-051225c0d0d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:00.332629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:00.333508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:00.334966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:00.334966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:00.334966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:00.334966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:00.334966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:00.334966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:00.334966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:00.334966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:00.334966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 328/647] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:00.334966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b52fe-e25b-4c97-9c41-051225c0d0d5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:00.375731 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01f760-9007-4a35-9d8d-2360dff7f5c5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:00.412523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710da4c-ea0c-459d-9824-ae00a17862c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:00.413145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710da4c-ea0c-459d-9824-ae00a17862c2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:00.413426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710da4c-ea0c-459d-9824-ae00a17862c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:00.413700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710da4c-ea0c-459d-9824-ae00a17862c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:00.414364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710da4c-ea0c-459d-9824-ae00a17862c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:00.414670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710da4c-ea0c-459d-9824-ae00a17862c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:00.416061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710da4c-ea0c-459d-9824-ae00a17862c2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:00.443659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 320/648] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:00.443659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710da4c-ea0c-459d-9824-ae00a17862c2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:00.497518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:00.553303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710da4c-ea0c-459d-9824-ae00a17862c2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:00.567582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b52fe-e25b-4c97-9c41-051225c0d0d5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:00.580497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:00.590811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282b00a-dc91-48a6-8c1f-78fce07eece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:00.591423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282b00a-dc91-48a6-8c1f-78fce07eece9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:00.591700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282b00a-dc91-48a6-8c1f-78fce07eece9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:00.591984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282b00a-dc91-48a6-8c1f-78fce07eece9 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:00.592877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282b00a-dc91-48a6-8c1f-78fce07eece9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:00.593186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282b00a-dc91-48a6-8c1f-78fce07eece9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:00.594548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282b00a-dc91-48a6-8c1f-78fce07eece9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:00.604699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 321/649] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:20:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:00.605816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282b00a-dc91-48a6-8c1f-78fce07eece9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:00.611413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ed3ac-8075-438b-a26d-01bfb74c96be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:00.612566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ed3ac-8075-438b-a26d-01bfb74c96be None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:00.612845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ed3ac-8075-438b-a26d-01bfb74c96be None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:00.613113 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ed3ac-8075-438b-a26d-01bfb74c96be None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:00.613846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ed3ac-8075-438b-a26d-01bfb74c96be None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:00.614147 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ed3ac-8075-438b-a26d-01bfb74c96be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:00.615571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ed3ac-8075-438b-a26d-01bfb74c96be None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:00.634237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 329/650] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 22:20:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:00.638756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ed3ac-8075-438b-a26d-01bfb74c96be None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:00.716194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282b00a-dc91-48a6-8c1f-78fce07eece9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:00.726180 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:00.746828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ed3ac-8075-438b-a26d-01bfb74c96be None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:00.749837 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c2079-8a21-40ac-90b7-81fb1f86e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:00.750376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c2079-8a21-40ac-90b7-81fb1f86e3ba None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:00.750650 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c2079-8a21-40ac-90b7-81fb1f86e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:00.750918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c2079-8a21-40ac-90b7-81fb1f86e3ba None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:00.751596 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c2079-8a21-40ac-90b7-81fb1f86e3ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:00.751896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c2079-8a21-40ac-90b7-81fb1f86e3ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:00.753326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c2079-8a21-40ac-90b7-81fb1f86e3ba None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:00.770624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 322/651] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:00.770624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c2079-8a21-40ac-90b7-81fb1f86e3ba None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:00.793165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb2051-d4ae-41ef-b94d-e690d272cd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:00.794352 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb2051-d4ae-41ef-b94d-e690d272cd08 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:00.794768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb2051-d4ae-41ef-b94d-e690d272cd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:00.795048 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb2051-d4ae-41ef-b94d-e690d272cd08 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:00.804618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb2051-d4ae-41ef-b94d-e690d272cd08 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:00.807012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb2051-d4ae-41ef-b94d-e690d272cd08 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-73cb2051-d4ae-41ef-b94d-e690d272cd08 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:00.824807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:00.825777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:00.826706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:00.826706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:00.826706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:00.826706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:00.826706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:00.826706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:00.826706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:00.826706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:00.826706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 22:20:00.826706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:00.828161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 330/652] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:20:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:00.828981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb2051-d4ae-41ef-b94d-e690d272cd08 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:00.858229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:00.881815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c2079-8a21-40ac-90b7-81fb1f86e3ba None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:00.920266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72d298-605e-49de-9316-07e8a1e99fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:00.920898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72d298-605e-49de-9316-07e8a1e99fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:00.921179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72d298-605e-49de-9316-07e8a1e99fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:00.921558 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72d298-605e-49de-9316-07e8a1e99fd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:00.932821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72d298-605e-49de-9316-07e8a1e99fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:00.934246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72d298-605e-49de-9316-07e8a1e99fd9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:00.937929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb2051-d4ae-41ef-b94d-e690d272cd08 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ed72d298-605e-49de-9316-07e8a1e99fd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:00.951765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:00.952962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:00.954267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:00.954267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:00.954267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:00.954267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 323/653] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:00.954267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72d298-605e-49de-9316-07e8a1e99fd9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:00.983242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:00.986504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f924a1-fb61-450e-bb27-e756142eaafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:00.987447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f924a1-fb61-450e-bb27-e756142eaafe None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:00.988005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f924a1-fb61-450e-bb27-e756142eaafe None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:00.988315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f924a1-fb61-450e-bb27-e756142eaafe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:01.060417 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72d298-605e-49de-9316-07e8a1e99fd9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:01.071070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f924a1-fb61-450e-bb27-e756142eaafe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:01.073245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f924a1-fb61-450e-bb27-e756142eaafe None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:01.098098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f861e2e-452d-46dd-8a8c-2f532948b043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:01.098813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861e2e-452d-46dd-8a8c-2f532948b043 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:01.099092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861e2e-452d-46dd-8a8c-2f532948b043 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:01.099366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861e2e-452d-46dd-8a8c-2f532948b043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:01.100121 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f861e2e-452d-46dd-8a8c-2f532948b043 None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=8effbd0670ef41f59b46feb9f0997554) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:01.100429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f861e2e-452d-46dd-8a8c-2f532948b043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:01.101836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f861e2e-452d-46dd-8a8c-2f532948b043 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:01.103177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:01.108043 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 324/654] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:00 2022] GET /identity/v3/role_assignments?user.id=8effbd0670ef41f59b46feb9f0997554&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:01.109181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f861e2e-452d-46dd-8a8c-2f532948b043 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:01.200903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 331/655] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:20:01.201883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f924a1-fb61-450e-bb27-e756142eaafe None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:01.228366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:01.245533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f861e2e-452d-46dd-8a8c-2f532948b043 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:01.285654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8a1af-7160-4b9b-b474-ae4500a410e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:01.286357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8a1af-7160-4b9b-b474-ae4500a410e6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:01.286707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8a1af-7160-4b9b-b474-ae4500a410e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:01.287092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8a1af-7160-4b9b-b474-ae4500a410e6 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:01.287850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8a1af-7160-4b9b-b474-ae4500a410e6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:01.288269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8a1af-7160-4b9b-b474-ae4500a410e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:01.290709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8a1af-7160-4b9b-b474-ae4500a410e6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:01.314487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 325/656] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 22:20:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:01.315221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8a1af-7160-4b9b-b474-ae4500a410e6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:01.323396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f924a1-fb61-450e-bb27-e756142eaafe None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:01.353855 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:01.364246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc1826-0ca5-49b6-956c-a8d75fae36f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:01.365457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1826-0ca5-49b6-956c-a8d75fae36f2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:01.365457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1826-0ca5-49b6-956c-a8d75fae36f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:01.365457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1826-0ca5-49b6-956c-a8d75fae36f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:01.371501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc1826-0ca5-49b6-956c-a8d75fae36f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:01.371501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc1826-0ca5-49b6-956c-a8d75fae36f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:01.371501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc1826-0ca5-49b6-956c-a8d75fae36f2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:01.382757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 332/657] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:01.384891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc1826-0ca5-49b6-956c-a8d75fae36f2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:01.437001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8a1af-7160-4b9b-b474-ae4500a410e6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:01.477248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08dc23-17c6-4d16-9880-24b92b6c72fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:01.478054 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08dc23-17c6-4d16-9880-24b92b6c72fe None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:01.478423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08dc23-17c6-4d16-9880-24b92b6c72fe None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:01.478791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08dc23-17c6-4d16-9880-24b92b6c72fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:01.493115 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08dc23-17c6-4d16-9880-24b92b6c72fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:01.495305 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08dc23-17c6-4d16-9880-24b92b6c72fe None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:01.504734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:01.505490 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc1826-0ca5-49b6-956c-a8d75fae36f2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-9e08dc23-17c6-4d16-9880-24b92b6c72fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:01.514187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:01.515173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:01.516292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:01.516292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:01.516292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:01.516292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 326/658] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:01.516292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08dc23-17c6-4d16-9880-24b92b6c72fe None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:01.547963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40399575-f578-4b99-920b-4acacd83eeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:01.548691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40399575-f578-4b99-920b-4acacd83eeae None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:01.549073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40399575-f578-4b99-920b-4acacd83eeae None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:01.549414 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40399575-f578-4b99-920b-4acacd83eeae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:01.612207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:01.624228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08dc23-17c6-4d16-9880-24b92b6c72fe None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:01.625784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40399575-f578-4b99-920b-4acacd83eeae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:01.627341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40399575-f578-4b99-920b-4acacd83eeae None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:01.659347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da595c11-cdac-4435-ab0d-6712faf92198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:01.659967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da595c11-cdac-4435-ab0d-6712faf92198 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:01.660276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da595c11-cdac-4435-ab0d-6712faf92198 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:01.660547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da595c11-cdac-4435-ab0d-6712faf92198 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:01.669461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da595c11-cdac-4435-ab0d-6712faf92198 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:01.670891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da595c11-cdac-4435-ab0d-6712faf92198 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-da595c11-cdac-4435-ab0d-6712faf92198 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:01.682199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:01.683550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:01.684543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:01.684543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:01.684543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:01.684543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:01.684543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:01.684543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:01.684543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:01.684543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:01.684543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 22:20:01.684543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:01.687822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 327/659] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:20:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:01.689047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909660b2-0d7e-4287-a1f1-f5cf43f4bdab None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:01.689551 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909660b2-0d7e-4287-a1f1-f5cf43f4bdab None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:01.689658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909660b2-0d7e-4287-a1f1-f5cf43f4bdab None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:01.692050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 328/660] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:01.692050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909660b2-0d7e-4287-a1f1-f5cf43f4bdab None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:01.736229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 333/661] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:20:01.737380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40399575-f578-4b99-920b-4acacd83eeae None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:01.759470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:01.807409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909660b2-0d7e-4287-a1f1-f5cf43f4bdab None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:01.830413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:01.843012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40399575-f578-4b99-920b-4acacd83eeae None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:01.843929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c87e-d891-4429-81d3-602f8f425eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:01.844622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c87e-d891-4429-81d3-602f8f425eaa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:01.844963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c87e-d891-4429-81d3-602f8f425eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:01.845942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c87e-d891-4429-81d3-602f8f425eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:01.845942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c87e-d891-4429-81d3-602f8f425eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:01.846329 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c87e-d891-4429-81d3-602f8f425eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:01.847740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c87e-d891-4429-81d3-602f8f425eaa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:01.860245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 329/662] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:01.865951 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91807c0f-0304-4696-a1b3-be7f976a02ad None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:01.866181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91807c0f-0304-4696-a1b3-be7f976a02ad None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:01.866453 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91807c0f-0304-4696-a1b3-be7f976a02ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:01.882041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a260e3-c0ba-477d-92ea-448df4e126af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:01.882726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a260e3-c0ba-477d-92ea-448df4e126af None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:01.883072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a260e3-c0ba-477d-92ea-448df4e126af None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:01.883407 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a260e3-c0ba-477d-92ea-448df4e126af None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:01.884320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a260e3-c0ba-477d-92ea-448df4e126af None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:01.884699 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a260e3-c0ba-477d-92ea-448df4e126af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:01.885558 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-91807c0f-0304-4696-a1b3-be7f976a02ad None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:01.886419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a260e3-c0ba-477d-92ea-448df4e126af None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:01.886931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-91807c0f-0304-4696-a1b3-be7f976a02ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:01.910561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 334/663] 10.222.0.30 () {56 vars in 1139 bytes} [Sun Aug 7 22:20:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:01.910561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a260e3-c0ba-477d-92ea-448df4e126af None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:01.961052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 330/664] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:20:01.962162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91807c0f-0304-4696-a1b3-be7f976a02ad None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:02.015565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:02.038560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a260e3-c0ba-477d-92ea-448df4e126af None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:02.072514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91807c0f-0304-4696-a1b3-be7f976a02ad None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:02.074410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365664d6-5784-4a95-9152-5cf1e2a6306d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:02.075119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365664d6-5784-4a95-9152-5cf1e2a6306d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:02.075459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365664d6-5784-4a95-9152-5cf1e2a6306d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:02.075803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365664d6-5784-4a95-9152-5cf1e2a6306d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:02.086106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365664d6-5784-4a95-9152-5cf1e2a6306d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:02.087609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365664d6-5784-4a95-9152-5cf1e2a6306d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:02.095707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-365664d6-5784-4a95-9152-5cf1e2a6306d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:02.104319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:02.110128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:02.111201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:02.111201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:02.111201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:02.111201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 335/665] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:02.111201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365664d6-5784-4a95-9152-5cf1e2a6306d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:02.114686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a8489-a161-4247-9413-05880645d355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:02.115336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a8489-a161-4247-9413-05880645d355 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:02.115612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a8489-a161-4247-9413-05880645d355 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:02.115881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a8489-a161-4247-9413-05880645d355 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:02.123763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a8489-a161-4247-9413-05880645d355 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:02.125185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a8489-a161-4247-9413-05880645d355 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-681a8489-a161-4247-9413-05880645d355 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:02.136351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:02.137280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:02.138175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:02.138175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:02.138175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:02.138175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:02.138175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:02.138175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:02.138175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:02.138175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:02.138175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 331/666] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:02.138175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a8489-a161-4247-9413-05880645d355 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:02.223415 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365664d6-5784-4a95-9152-5cf1e2a6306d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:02.245835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a8489-a161-4247-9413-05880645d355 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:02.254242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:02.259476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa40cb-60fd-4023-a306-4652451caafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:02.260188 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa40cb-60fd-4023-a306-4652451caafc None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:02.260475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa40cb-60fd-4023-a306-4652451caafc None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:02.260747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa40cb-60fd-4023-a306-4652451caafc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:02.283503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6cb49-146a-4eb8-bfec-8b0afe4b7483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:02.284388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6cb49-146a-4eb8-bfec-8b0afe4b7483 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:02.284730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6cb49-146a-4eb8-bfec-8b0afe4b7483 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:02.285060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6cb49-146a-4eb8-bfec-8b0afe4b7483 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:02.296412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6cb49-146a-4eb8-bfec-8b0afe4b7483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-5ac6cb49-146a-4eb8-bfec-8b0afe4b7483 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:02.300666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:02.301800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:20:02.302829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:02.306632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 332/667] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:20:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:02.308065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6cb49-146a-4eb8-bfec-8b0afe4b7483 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:02.332787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:02.337516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa40cb-60fd-4023-a306-4652451caafc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:02.339185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa40cb-60fd-4023-a306-4652451caafc None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:02.416977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6cb49-146a-4eb8-bfec-8b0afe4b7483 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:02.453062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8d8c1-40f4-41e5-8e14-9305c7f2fbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:02.458173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 336/668] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:20:02.458321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8d8c1-40f4-41e5-8e14-9305c7f2fbaf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:02.458321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8d8c1-40f4-41e5-8e14-9305c7f2fbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:02.458321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8d8c1-40f4-41e5-8e14-9305c7f2fbaf None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:02.458597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa40cb-60fd-4023-a306-4652451caafc None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:02.458685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8d8c1-40f4-41e5-8e14-9305c7f2fbaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:02.458685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8d8c1-40f4-41e5-8e14-9305c7f2fbaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:02.458685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8d8c1-40f4-41e5-8e14-9305c7f2fbaf None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:02.472302 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 333/669] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:02.477425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8d8c1-40f4-41e5-8e14-9305c7f2fbaf None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:02.507306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:02.562629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:02.566899 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa40cb-60fd-4023-a306-4652451caafc None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:02.583170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8d8c1-40f4-41e5-8e14-9305c7f2fbaf None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:02.603384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57915547-fad0-4c6e-abea-965f7b717bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:02.604683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57915547-fad0-4c6e-abea-965f7b717bca None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:02.605444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57915547-fad0-4c6e-abea-965f7b717bca None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:02.605844 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57915547-fad0-4c6e-abea-965f7b717bca None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:02.606635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57915547-fad0-4c6e-abea-965f7b717bca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:02.607042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57915547-fad0-4c6e-abea-965f7b717bca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:02.608594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57915547-fad0-4c6e-abea-965f7b717bca None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:02.618832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7bb83-bc1f-44a0-8d12-0e59b4e25d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:02.619432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7bb83-bc1f-44a0-8d12-0e59b4e25d01 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:02.620637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7bb83-bc1f-44a0-8d12-0e59b4e25d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:02.620980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7bb83-bc1f-44a0-8d12-0e59b4e25d01 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:02.626646 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 337/670] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:02.626783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7bb83-bc1f-44a0-8d12-0e59b4e25d01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:02.626783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7bb83-bc1f-44a0-8d12-0e59b4e25d01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:02.626946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57915547-fad0-4c6e-abea-965f7b717bca None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:02.627030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7bb83-bc1f-44a0-8d12-0e59b4e25d01 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:02.634023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 334/671] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:20:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:02.639052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7bb83-bc1f-44a0-8d12-0e59b4e25d01 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:02.736280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:02.737234 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57915547-fad0-4c6e-abea-965f7b717bca None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:02.749711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7bb83-bc1f-44a0-8d12-0e59b4e25d01 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:02.773030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1640b-520a-43fb-b943-c8dc4ff9f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:02.773946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1640b-520a-43fb-b943-c8dc4ff9f7fe None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:02.774221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1640b-520a-43fb-b943-c8dc4ff9f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:02.774658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1640b-520a-43fb-b943-c8dc4ff9f7fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:02.785780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1640b-520a-43fb-b943-c8dc4ff9f7fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:02.787217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1640b-520a-43fb-b943-c8dc4ff9f7fe None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:02.789110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7ea8f-3473-47c8-8fc9-19b94fd4221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:02.790033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7ea8f-3473-47c8-8fc9-19b94fd4221b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:02.790312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7ea8f-3473-47c8-8fc9-19b94fd4221b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:02.790579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7ea8f-3473-47c8-8fc9-19b94fd4221b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:02.799384 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7ea8f-3473-47c8-8fc9-19b94fd4221b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:02.800833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7ea8f-3473-47c8-8fc9-19b94fd4221b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-29e1640b-520a-43fb-b943-c8dc4ff9f7fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:02.802931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:02.803861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:02.805718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:02.805718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:02.805718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:02.805718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 338/672] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:02.805718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1640b-520a-43fb-b943-c8dc4ff9f7fe None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-a5c7ea8f-3473-47c8-8fc9-19b94fd4221b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:02.817426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:02.818337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:02.819246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:02.819246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:02.819246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:02.819246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:02.819246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:02.819246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:02.819246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:02.819246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:02.819246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 335/673] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:02.819742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7ea8f-3473-47c8-8fc9-19b94fd4221b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:02.825010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:02.923506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7ea8f-3473-47c8-8fc9-19b94fd4221b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:02.932675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1640b-520a-43fb-b943-c8dc4ff9f7fe None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:02.959668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe37363-dccd-403d-bffb-76631fcbcbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:02.960326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe37363-dccd-403d-bffb-76631fcbcbad None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:02.960609 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe37363-dccd-403d-bffb-76631fcbcbad None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:02.960884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe37363-dccd-403d-bffb-76631fcbcbad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:02.972659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe37363-dccd-403d-bffb-76631fcbcbad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:02.974378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe37363-dccd-403d-bffb-76631fcbcbad None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:02.975740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824c1b8-83e4-4432-90f2-0f0291755cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:02.976388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824c1b8-83e4-4432-90f2-0f0291755cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:02.976670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824c1b8-83e4-4432-90f2-0f0291755cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:02.976946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824c1b8-83e4-4432-90f2-0f0291755cf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:02.977689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824c1b8-83e4-4432-90f2-0f0291755cf9 None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=cfb2129c006f4ddeba7d128a8d43d5de) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:02.977999 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824c1b8-83e4-4432-90f2-0f0291755cf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:02.980386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824c1b8-83e4-4432-90f2-0f0291755cf9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:02.985741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:02.996125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 339/674] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:02 2022] GET /identity/v3/role_assignments?user.id=cfb2129c006f4ddeba7d128a8d43d5de&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:02.996125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824c1b8-83e4-4432-90f2-0f0291755cf9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ebe37363-dccd-403d-bffb-76631fcbcbad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:03.002627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:03.003803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:03.004913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:03.004913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:03.004913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:03.004913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 336/675] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:03.006042 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe37363-dccd-403d-bffb-76631fcbcbad None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:03.096320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:03.125664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe37363-dccd-403d-bffb-76631fcbcbad None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:03.126834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824c1b8-83e4-4432-90f2-0f0291755cf9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:03.166238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e81f4-0a57-4fa0-9df0-e28ed9a35836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:03.167457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e81f4-0a57-4fa0-9df0-e28ed9a35836 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:03.167568 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa1636-337f-4560-9218-63273c371a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:03.167669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e81f4-0a57-4fa0-9df0-e28ed9a35836 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:03.167923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e81f4-0a57-4fa0-9df0-e28ed9a35836 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:03.168114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa1636-337f-4560-9218-63273c371a70 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:03.168404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa1636-337f-4560-9218-63273c371a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:03.168641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e81f4-0a57-4fa0-9df0-e28ed9a35836 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:03.168747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa1636-337f-4560-9218-63273c371a70 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:03.168947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e81f4-0a57-4fa0-9df0-e28ed9a35836 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:03.169327 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa1636-337f-4560-9218-63273c371a70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:03.169634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa1636-337f-4560-9218-63273c371a70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:03.170358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e81f4-0a57-4fa0-9df0-e28ed9a35836 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:03.170995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa1636-337f-4560-9218-63273c371a70 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:03.184259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 337/676] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:03.185109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e81f4-0a57-4fa0-9df0-e28ed9a35836 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:03.185857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 340/677] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:03.190847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa1636-337f-4560-9218-63273c371a70 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:03.256751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:03.308207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e81f4-0a57-4fa0-9df0-e28ed9a35836 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:03.322171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa1636-337f-4560-9218-63273c371a70 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:03.347001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff63bd-6cf0-4308-ac28-adaac5799011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:03.347643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff63bd-6cf0-4308-ac28-adaac5799011 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:03.348055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff63bd-6cf0-4308-ac28-adaac5799011 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:03.348349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff63bd-6cf0-4308-ac28-adaac5799011 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:03.349091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff63bd-6cf0-4308-ac28-adaac5799011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:03.349397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff63bd-6cf0-4308-ac28-adaac5799011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:03.350775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff63bd-6cf0-4308-ac28-adaac5799011 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:03.359184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4af915-d78c-4e43-ad1e-1cf9cd3e20fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:03.359817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4af915-d78c-4e43-ad1e-1cf9cd3e20fe None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:03.360117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4af915-d78c-4e43-ad1e-1cf9cd3e20fe None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:03.360389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4af915-d78c-4e43-ad1e-1cf9cd3e20fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:03.362254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:03.365835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 338/678] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:03.367019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff63bd-6cf0-4308-ac28-adaac5799011 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:03.373348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4af915-d78c-4e43-ad1e-1cf9cd3e20fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:03.374861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4af915-d78c-4e43-ad1e-1cf9cd3e20fe None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-7d4af915-d78c-4e43-ad1e-1cf9cd3e20fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:03.387117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:03.399308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:03.400228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:03.400228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:03.400228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:03.400228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:03.400228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:03.400228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:03.400228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:03.400228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:03.400228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 341/679] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:03.400228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c60c4e-72f7-43a7-8c7d-0f4fbdac76b4 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:03.400228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c60c4e-72f7-43a7-8c7d-0f4fbdac76b4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:03.401458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c60c4e-72f7-43a7-8c7d-0f4fbdac76b4 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:03.401458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 342/680] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:03.401458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c60c4e-72f7-43a7-8c7d-0f4fbdac76b4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:03.488669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff63bd-6cf0-4308-ac28-adaac5799011 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:03.505250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c60c4e-72f7-43a7-8c7d-0f4fbdac76b4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:03.511027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:03.527335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436f021-2fc3-4224-8fd0-c7f54278413f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:03.527948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436f021-2fc3-4224-8fd0-c7f54278413f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:03.528261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436f021-2fc3-4224-8fd0-c7f54278413f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:03.528540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436f021-2fc3-4224-8fd0-c7f54278413f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:03.529277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436f021-2fc3-4224-8fd0-c7f54278413f None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=407b8483d4f34be19d2c7c7b6013b313) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:03.529588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436f021-2fc3-4224-8fd0-c7f54278413f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:03.531044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436f021-2fc3-4224-8fd0-c7f54278413f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:03.542815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 339/681] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:03 2022] GET /identity/v3/role_assignments?user.id=407b8483d4f34be19d2c7c7b6013b313&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:03.542815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436f021-2fc3-4224-8fd0-c7f54278413f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:03.558515 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a78a3-1bfc-494c-95a2-273f5398a1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:03.559187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a78a3-1bfc-494c-95a2-273f5398a1cd None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:03.559469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a78a3-1bfc-494c-95a2-273f5398a1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:03.559879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a78a3-1bfc-494c-95a2-273f5398a1cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:03.571220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a78a3-1bfc-494c-95a2-273f5398a1cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:03.572937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a78a3-1bfc-494c-95a2-273f5398a1cd None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-844a78a3-1bfc-494c-95a2-273f5398a1cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:03.585676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:03.586821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:03.587776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:03.587776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:03.587776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:03.587776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:03.587776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:03.587776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:03.587776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:03.587776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:03.587776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:20:03.587776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:03.592586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 343/682] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:03.593851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec7c10-97b4-4e88-94a4-ba8bfd1776c6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:03.594149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec7c10-97b4-4e88-94a4-ba8bfd1776c6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:03.594429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec7c10-97b4-4e88-94a4-ba8bfd1776c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:03.620452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-94ec7c10-97b4-4e88-94a4-ba8bfd1776c6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:03.622552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-94ec7c10-97b4-4e88-94a4-ba8bfd1776c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:03.671262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436f021-2fc3-4224-8fd0-c7f54278413f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:03.691789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:03.717624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 344/683] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:20:03.717624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec7c10-97b4-4e88-94a4-ba8bfd1776c6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:03.721529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85231ed3-9535-4847-a6c3-963736371a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:03.722255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85231ed3-9535-4847-a6c3-963736371a1a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:03.722535 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85231ed3-9535-4847-a6c3-963736371a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:03.722811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85231ed3-9535-4847-a6c3-963736371a1a None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:03.723526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85231ed3-9535-4847-a6c3-963736371a1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:03.723834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85231ed3-9535-4847-a6c3-963736371a1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:03.728066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85231ed3-9535-4847-a6c3-963736371a1a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:03.738716 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 340/684] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:03.744261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85231ed3-9535-4847-a6c3-963736371a1a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:03.795228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:03.851018 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec7c10-97b4-4e88-94a4-ba8bfd1776c6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:03.864577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85231ed3-9535-4847-a6c3-963736371a1a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:03.888834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7826b-6b0b-4456-9ab2-8048f5c8419e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:03.889442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7826b-6b0b-4456-9ab2-8048f5c8419e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:03.889718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7826b-6b0b-4456-9ab2-8048f5c8419e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:03.889989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7826b-6b0b-4456-9ab2-8048f5c8419e None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:03.890820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7826b-6b0b-4456-9ab2-8048f5c8419e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:03.891019 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7826b-6b0b-4456-9ab2-8048f5c8419e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:03.892511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7826b-6b0b-4456-9ab2-8048f5c8419e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:03.915322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 345/685] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 22:20:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:03.915322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7826b-6b0b-4456-9ab2-8048f5c8419e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:03.919491 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5919cc7-8777-4717-8165-d4f70f47a12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:03.920347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5919cc7-8777-4717-8165-d4f70f47a12e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:03.920778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5919cc7-8777-4717-8165-d4f70f47a12e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:03.921201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5919cc7-8777-4717-8165-d4f70f47a12e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:03.936419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5919cc7-8777-4717-8165-d4f70f47a12e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c5919cc7-8777-4717-8165-d4f70f47a12e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:03.947280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:03.949277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:03.950318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:03.957548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 341/686] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:03.957548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5919cc7-8777-4717-8165-d4f70f47a12e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:03.971632 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:04.053304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7826b-6b0b-4456-9ab2-8048f5c8419e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:04.064124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5919cc7-8777-4717-8165-d4f70f47a12e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:04.067349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:04.100243 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f5153-3837-4418-bc70-89e52b72bf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:04.100865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51720021-b7fe-4374-9908-927bd9c3fbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:04.101360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720021-b7fe-4374-9908-927bd9c3fbea None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:04.102142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f5153-3837-4418-bc70-89e52b72bf8d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:04.102257 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720021-b7fe-4374-9908-927bd9c3fbea None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:04.102524 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720021-b7fe-4374-9908-927bd9c3fbea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:04.102932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f5153-3837-4418-bc70-89e52b72bf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:04.103601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f5153-3837-4418-bc70-89e52b72bf8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:04.105311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f5153-3837-4418-bc70-89e52b72bf8d None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=baffd4f6a8ad4a6e91d8d9c61cc1dce0) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:04.105774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f5153-3837-4418-bc70-89e52b72bf8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:04.108382 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f5153-3837-4418-bc70-89e52b72bf8d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:04.117138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 346/687] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:03 2022] GET /identity/v3/role_assignments?user.id=baffd4f6a8ad4a6e91d8d9c61cc1dce0&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:04.118359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f5153-3837-4418-bc70-89e52b72bf8d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:04.248931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51720021-b7fe-4374-9908-927bd9c3fbea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:04.250423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51720021-b7fe-4374-9908-927bd9c3fbea None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:04.301382 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f5153-3837-4418-bc70-89e52b72bf8d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:04.311613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:04.348647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00faf943-cc34-4e19-9bd7-3c7734ee164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:04.350160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00faf943-cc34-4e19-9bd7-3c7734ee164b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:04.350747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00faf943-cc34-4e19-9bd7-3c7734ee164b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:04.351300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00faf943-cc34-4e19-9bd7-3c7734ee164b None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:04.352198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00faf943-cc34-4e19-9bd7-3c7734ee164b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:04.352678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00faf943-cc34-4e19-9bd7-3c7734ee164b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:04.355221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00faf943-cc34-4e19-9bd7-3c7734ee164b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:04.367254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 347/688] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:20:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:04.373440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00faf943-cc34-4e19-9bd7-3c7734ee164b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:04.405362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 342/689] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:20:04.406776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51720021-b7fe-4374-9908-927bd9c3fbea None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:04.409692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:04.498519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00faf943-cc34-4e19-9bd7-3c7734ee164b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:04.522371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51720021-b7fe-4374-9908-927bd9c3fbea None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:04.533505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91714651-e657-4dbb-9f60-44b369c45fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:04.534474 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91714651-e657-4dbb-9f60-44b369c45fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:04.534946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91714651-e657-4dbb-9f60-44b369c45fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:04.535420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91714651-e657-4dbb-9f60-44b369c45fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:04.546966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91714651-e657-4dbb-9f60-44b369c45fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:04.548891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91714651-e657-4dbb-9f60-44b369c45fe0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:04.558559 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bec44a-2998-4819-bee6-65ca532614fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:04.559189 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bec44a-2998-4819-bee6-65ca532614fb None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:04.559467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bec44a-2998-4819-bee6-65ca532614fb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:04.559738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bec44a-2998-4819-bee6-65ca532614fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-91714651-e657-4dbb-9f60-44b369c45fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:04.565597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:04.566507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:04.567429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:04.567429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:04.567429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:04.569619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 348/690] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:04.570678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bec44a-2998-4819-bee6-65ca532614fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:04.574320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bec44a-2998-4819-bee6-65ca532614fb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:04.575381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91714651-e657-4dbb-9f60-44b369c45fe0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-10bec44a-2998-4819-bee6-65ca532614fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:04.590201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:04.591213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:04.592111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:04.592111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:04.592111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:04.592111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 343/691] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:04.597413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bec44a-2998-4819-bee6-65ca532614fb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:04.632061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:04.657793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:04.702726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bec44a-2998-4819-bee6-65ca532614fb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:04.739898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86de89-2a58-4b85-b90a-f5d6619739da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:04.740539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86de89-2a58-4b85-b90a-f5d6619739da None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:04.740816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86de89-2a58-4b85-b90a-f5d6619739da None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:04.741086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86de89-2a58-4b85-b90a-f5d6619739da None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:04.741737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86de89-2a58-4b85-b90a-f5d6619739da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:04.742042 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86de89-2a58-4b85-b90a-f5d6619739da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:04.743402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86de89-2a58-4b85-b90a-f5d6619739da None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:04.757561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 344/692] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:04.763386 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86de89-2a58-4b85-b90a-f5d6619739da None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:04.832488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91714651-e657-4dbb-9f60-44b369c45fe0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:04.881523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86de89-2a58-4b85-b90a-f5d6619739da None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:04.884195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5aaca-c5b6-4367-89b8-fec5830bb745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:04.884813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5aaca-c5b6-4367-89b8-fec5830bb745 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:04.885733 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5aaca-c5b6-4367-89b8-fec5830bb745 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:04.886006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5aaca-c5b6-4367-89b8-fec5830bb745 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:04.886649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5aaca-c5b6-4367-89b8-fec5830bb745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:04.886954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5aaca-c5b6-4367-89b8-fec5830bb745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:04.888385 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5aaca-c5b6-4367-89b8-fec5830bb745 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:04.900748 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:04.901108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 349/693] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:04.906634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5aaca-c5b6-4367-89b8-fec5830bb745 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:04.920328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012223e6-4540-4c5f-869a-ae9cbd63d227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:04.923116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012223e6-4540-4c5f-869a-ae9cbd63d227 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:04.923116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012223e6-4540-4c5f-869a-ae9cbd63d227 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:04.923116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012223e6-4540-4c5f-869a-ae9cbd63d227 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:04.928785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012223e6-4540-4c5f-869a-ae9cbd63d227 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:04.930312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012223e6-4540-4c5f-869a-ae9cbd63d227 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-012223e6-4540-4c5f-869a-ae9cbd63d227 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:04.945242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:04.946161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:04.948866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:04.948866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:04.948866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:04.948866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:04.948866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:04.948866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:04.948866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:04.948866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:04.948866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 345/694] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:04.957282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012223e6-4540-4c5f-869a-ae9cbd63d227 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:04.992450 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:05.013057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5aaca-c5b6-4367-89b8-fec5830bb745 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:05.047619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c12e0-30c7-4952-8a7b-532c42006472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:05.048293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c12e0-30c7-4952-8a7b-532c42006472 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:05.048573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c12e0-30c7-4952-8a7b-532c42006472 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:05.048846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c12e0-30c7-4952-8a7b-532c42006472 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:05.056124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c12e0-30c7-4952-8a7b-532c42006472 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:05.058094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c12e0-30c7-4952-8a7b-532c42006472 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b26c12e0-30c7-4952-8a7b-532c42006472 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:05.069840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:05.072055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:05.074756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:05.074756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:05.074756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:05.074756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:05.074756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:05.074756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:05.074756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:05.074756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:05.074756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:05.074756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:05.074756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 350/695] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 22:20:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:05.080541 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c12e0-30c7-4952-8a7b-532c42006472 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:05.136866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012223e6-4540-4c5f-869a-ae9cbd63d227 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:05.173797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a4b3b-f618-4ca3-8abf-a6f8d5cf3aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:05.174507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a4b3b-f618-4ca3-8abf-a6f8d5cf3aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:05.174787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a4b3b-f618-4ca3-8abf-a6f8d5cf3aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:05.175057 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a4b3b-f618-4ca3-8abf-a6f8d5cf3aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:05.175752 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a4b3b-f618-4ca3-8abf-a6f8d5cf3aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:05.176093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a4b3b-f618-4ca3-8abf-a6f8d5cf3aa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:05.177707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a4b3b-f618-4ca3-8abf-a6f8d5cf3aa7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:05.179858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c12e0-30c7-4952-8a7b-532c42006472 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:05.190471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 346/696] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:05.196256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a4b3b-f618-4ca3-8abf-a6f8d5cf3aa7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:05.215393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbc2fc-d2c7-4701-9896-a95b523a0938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:05.216420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbc2fc-d2c7-4701-9896-a95b523a0938 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:05.216420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbc2fc-d2c7-4701-9896-a95b523a0938 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:05.216763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbc2fc-d2c7-4701-9896-a95b523a0938 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:05.217341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbc2fc-d2c7-4701-9896-a95b523a0938 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:05.217790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbc2fc-d2c7-4701-9896-a95b523a0938 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:05.219225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbc2fc-d2c7-4701-9896-a95b523a0938 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:05.229998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:05.238624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 351/697] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 22:20:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:05.244068 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbc2fc-d2c7-4701-9896-a95b523a0938 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:05.255242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:05.334868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a4b3b-f618-4ca3-8abf-a6f8d5cf3aa7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:05.360137 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbc2fc-d2c7-4701-9896-a95b523a0938 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:05.396088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628dc0f-e731-4248-b0b7-6c9981d6518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:05.396637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628dc0f-e731-4248-b0b7-6c9981d6518e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:05.396916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628dc0f-e731-4248-b0b7-6c9981d6518e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:05.397188 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628dc0f-e731-4248-b0b7-6c9981d6518e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:05.426815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3064d-8f6f-46f5-bb78-46a5dd69fe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:05.427808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3064d-8f6f-46f5-bb78-46a5dd69fe89 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:05.428327 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3064d-8f6f-46f5-bb78-46a5dd69fe89 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:05.428833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3064d-8f6f-46f5-bb78-46a5dd69fe89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:05.429781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3064d-8f6f-46f5-bb78-46a5dd69fe89 None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=e86991778c2b47c58f10d9d9e3ca5e82) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:05.430308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3064d-8f6f-46f5-bb78-46a5dd69fe89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:05.432190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3064d-8f6f-46f5-bb78-46a5dd69fe89 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:05.454582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 347/698] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:05 2022] GET /identity/v3/role_assignments?user.id=e86991778c2b47c58f10d9d9e3ca5e82&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:05.469517 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628dc0f-e731-4248-b0b7-6c9981d6518e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:05.471189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628dc0f-e731-4248-b0b7-6c9981d6518e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:05.476479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:05.585278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 352/699] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:20:05.591143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3064d-8f6f-46f5-bb78-46a5dd69fe89 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:05.693814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cfdd7-886d-4417-ab81-520332610d9e None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:05.694010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cfdd7-886d-4417-ab81-520332610d9e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:05.694280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cfdd7-886d-4417-ab81-520332610d9e None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:05.698481 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 353/700] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:05.702721 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:05.706140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3064d-8f6f-46f5-bb78-46a5dd69fe89 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:05.706392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d518c-b1cc-42b0-bbe1-3b5ed3c40ef1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:05.706687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d518c-b1cc-42b0-bbe1-3b5ed3c40ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:05.706967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d518c-b1cc-42b0-bbe1-3b5ed3c40ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:05.726618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f76d518c-b1cc-42b0-bbe1-3b5ed3c40ef1 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:05.727824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f76d518c-b1cc-42b0-bbe1-3b5ed3c40ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:05.747155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c11520-7547-4987-a451-d2b8f1dc43e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:05.747791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c11520-7547-4987-a451-d2b8f1dc43e4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:05.748092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c11520-7547-4987-a451-d2b8f1dc43e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:05.748364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c11520-7547-4987-a451-d2b8f1dc43e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:05.765841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c11520-7547-4987-a451-d2b8f1dc43e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:05.767324 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c11520-7547-4987-a451-d2b8f1dc43e4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-69c11520-7547-4987-a451-d2b8f1dc43e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:05.786161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:05.787267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:05.789169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:05.789169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:05.789169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:05.789169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 348/701] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:05.793903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c11520-7547-4987-a451-d2b8f1dc43e4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:05.811358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 354/702] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 22:20:05.839413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d518c-b1cc-42b0-bbe1-3b5ed3c40ef1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:05.871898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:05.912371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c11520-7547-4987-a451-d2b8f1dc43e4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:05.965835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20882e57-6762-4052-a77f-f1284e899f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:05.966355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20882e57-6762-4052-a77f-f1284e899f92 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:05.966637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20882e57-6762-4052-a77f-f1284e899f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:05.966875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20882e57-6762-4052-a77f-f1284e899f92 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:05.967511 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20882e57-6762-4052-a77f-f1284e899f92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:05.967815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20882e57-6762-4052-a77f-f1284e899f92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:05.968519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d518c-b1cc-42b0-bbe1-3b5ed3c40ef1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:05.969215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20882e57-6762-4052-a77f-f1284e899f92 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:05.984905 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 349/703] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:05.986052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb333c2-ca80-4296-87f8-9c04957e0a38 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:05.986396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb333c2-ca80-4296-87f8-9c04957e0a38 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:05.986696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb333c2-ca80-4296-87f8-9c04957e0a38 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:05.987944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 350/704] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:05.992846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb333c2-ca80-4296-87f8-9c04957e0a38 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:06.012402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:06.013638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf3e23-37da-4a8d-8fad-358a8e8c6f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:06.014564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf3e23-37da-4a8d-8fad-358a8e8c6f79 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.014846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf3e23-37da-4a8d-8fad-358a8e8c6f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.015123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf3e23-37da-4a8d-8fad-358a8e8c6f79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.029998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf3e23-37da-4a8d-8fad-358a8e8c6f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:06.031471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf3e23-37da-4a8d-8fad-358a8e8c6f79 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-aecf3e23-37da-4a8d-8fad-358a8e8c6f79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:06.050990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:06.052076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:06.052984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:06.052984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:06.052984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:06.053934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 355/705] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:06.056780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22fa88-dd2e-456f-af59-30bfb5c88eb8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.058166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22fa88-dd2e-456f-af59-30bfb5c88eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.058700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22fa88-dd2e-456f-af59-30bfb5c88eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.085685 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8a22fa88-dd2e-456f-af59-30bfb5c88eb8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:06.087262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8a22fa88-dd2e-456f-af59-30bfb5c88eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:06.128671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb333c2-ca80-4296-87f8-9c04957e0a38 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:06.165550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8111a-f240-4923-9c52-3fcd4eb16a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:06.166754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8111a-f240-4923-9c52-3fcd4eb16a1d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.167053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8111a-f240-4923-9c52-3fcd4eb16a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.167300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8111a-f240-4923-9c52-3fcd4eb16a1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.178780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 356/706] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:20:06.178780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22fa88-dd2e-456f-af59-30bfb5c88eb8 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:06.180428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8111a-f240-4923-9c52-3fcd4eb16a1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:06.182792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8111a-f240-4923-9c52-3fcd4eb16a1d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:06.199560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f4e8111a-f240-4923-9c52-3fcd4eb16a1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:06.203761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:06.204697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:06.205600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:06.205600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:06.205600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:06.205600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:06.205600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:06.205600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:06.205600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:06.205600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:06.205600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 351/707] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:06.206101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8111a-f240-4923-9c52-3fcd4eb16a1d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:06.303816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:06.321951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22fa88-dd2e-456f-af59-30bfb5c88eb8 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:06.346981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8111a-f240-4923-9c52-3fcd4eb16a1d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:06.368505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fbf90-7207-405c-ae42-dc0fb8260e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:06.369122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fbf90-7207-405c-ae42-dc0fb8260e57 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.369616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fbf90-7207-405c-ae42-dc0fb8260e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.369893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fbf90-7207-405c-ae42-dc0fb8260e57 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.370807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fbf90-7207-405c-ae42-dc0fb8260e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:06.371409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fbf90-7207-405c-ae42-dc0fb8260e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:06.372824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fbf90-7207-405c-ae42-dc0fb8260e57 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:06.392190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 357/708] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:06.393081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fbf90-7207-405c-ae42-dc0fb8260e57 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:06.395269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4726c5-0766-465c-9092-69a5afb83926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:06.395939 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4726c5-0766-465c-9092-69a5afb83926 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.396512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4726c5-0766-465c-9092-69a5afb83926 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.396789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4726c5-0766-465c-9092-69a5afb83926 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.398536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4726c5-0766-465c-9092-69a5afb83926 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:06.399925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4726c5-0766-465c-9092-69a5afb83926 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-0b4726c5-0766-465c-9092-69a5afb83926 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:06.415466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:06.418439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:06.420576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 352/709] 10.222.0.30 () {56 vars in 1129 bytes} [Sun Aug 7 22:20:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:06.427765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4726c5-0766-465c-9092-69a5afb83926 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:06.484156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:06.518853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fbf90-7207-405c-ae42-dc0fb8260e57 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:06.560431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985efe00-9f02-4f80-b74d-dba706f45a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:06.560973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985efe00-9f02-4f80-b74d-dba706f45a84 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.561242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985efe00-9f02-4f80-b74d-dba706f45a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.561572 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985efe00-9f02-4f80-b74d-dba706f45a84 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.562273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985efe00-9f02-4f80-b74d-dba706f45a84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:06.562585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985efe00-9f02-4f80-b74d-dba706f45a84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:06.564352 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985efe00-9f02-4f80-b74d-dba706f45a84 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:06.567743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4726c5-0766-465c-9092-69a5afb83926 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:06.577650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 358/710] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:06.578567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985efe00-9f02-4f80-b74d-dba706f45a84 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:06.615098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196358d7-1114-4850-9866-07107a0126f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:06.616420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196358d7-1114-4850-9866-07107a0126f2 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.617113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196358d7-1114-4850-9866-07107a0126f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.617553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196358d7-1114-4850-9866-07107a0126f2 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.618356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196358d7-1114-4850-9866-07107a0126f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:06.621257 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196358d7-1114-4850-9866-07107a0126f2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:06.632208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:06.690877 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985efe00-9f02-4f80-b74d-dba706f45a84 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:06.720522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:06.726715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2914e4-3be4-455f-af64-fdd4d67840e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:06.727332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2914e4-3be4-455f-af64-fdd4d67840e1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.727742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2914e4-3be4-455f-af64-fdd4d67840e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.728053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2914e4-3be4-455f-af64-fdd4d67840e1 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.728713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2914e4-3be4-455f-af64-fdd4d67840e1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:06.729230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2914e4-3be4-455f-af64-fdd4d67840e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:06.730655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2914e4-3be4-455f-af64-fdd4d67840e1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:06.740477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 353/711] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:20:06 2022] POST /identity/v3/users => generated 254 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:06.741919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196358d7-1114-4850-9866-07107a0126f2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:06.742682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 359/712] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 22:20:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:06.745349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadf7d-0b1a-4a3c-b0eb-4c365a2ef460 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.745349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadf7d-0b1a-4a3c-b0eb-4c365a2ef460 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.745349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadf7d-0b1a-4a3c-b0eb-4c365a2ef460 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.746819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 360/713] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:06.751687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadf7d-0b1a-4a3c-b0eb-4c365a2ef460 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:06.871386 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196358d7-1114-4850-9866-07107a0126f2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:06.871785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadf7d-0b1a-4a3c-b0eb-4c365a2ef460 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:06.910031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba41d43-6af2-483d-b5a2-c0d81f9da3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:06.910645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba41d43-6af2-483d-b5a2-c0d81f9da3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.910921 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba41d43-6af2-483d-b5a2-c0d81f9da3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.911190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba41d43-6af2-483d-b5a2-c0d81f9da3b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.911928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba41d43-6af2-483d-b5a2-c0d81f9da3b9 None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=8effbd0670ef41f59b46feb9f0997554) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:06.912265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba41d43-6af2-483d-b5a2-c0d81f9da3b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:06.913709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba41d43-6af2-483d-b5a2-c0d81f9da3b9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:06.923444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 354/714] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:06 2022] GET /identity/v3/role_assignments?user.id=8effbd0670ef41f59b46feb9f0997554&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=108664305d2546949f7327b65d4cb667 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:06.923628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b38822-14da-4654-8b03-ab24b556f6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:06.924286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b38822-14da-4654-8b03-ab24b556f6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.924568 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b38822-14da-4654-8b03-ab24b556f6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.924839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b38822-14da-4654-8b03-ab24b556f6c8 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.925477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b38822-14da-4654-8b03-ab24b556f6c8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:06.925782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b38822-14da-4654-8b03-ab24b556f6c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:06.926502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c36295-247a-4041-9e74-ce51ec508085 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:06.927024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c36295-247a-4041-9e74-ce51ec508085 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:06.927238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b38822-14da-4654-8b03-ab24b556f6c8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:06.927803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c36295-247a-4041-9e74-ce51ec508085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:06.934379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 361/715] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 22:20:06 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:06.952794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-55c36295-247a-4041-9e74-ce51ec508085 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:06.954258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-55c36295-247a-4041-9e74-ce51ec508085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:07.020774 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:07.041715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 355/716] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:20:07.065388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b38822-14da-4654-8b03-ab24b556f6c8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:07.137085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:07.187886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b38822-14da-4654-8b03-ab24b556f6c8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:07.237804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2949f8-de64-492c-8e8a-f3e48c84d631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:07.238446 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2949f8-de64-492c-8e8a-f3e48c84d631 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:07.238724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2949f8-de64-492c-8e8a-f3e48c84d631 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:07.239004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2949f8-de64-492c-8e8a-f3e48c84d631 None admin] PATH_INFO: `/v3/services/network` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:07.239602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2949f8-de64-492c-8e8a-f3e48c84d631 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:07.241010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2949f8-de64-492c-8e8a-f3e48c84d631 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-3a2949f8-de64-492c-8e8a-f3e48c84d631 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:07.254130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:07.255148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:07.255984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 362/717] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:07 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:07.261782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c36295-247a-4041-9e74-ce51ec508085 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:07.381651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c36295-247a-4041-9e74-ce51ec508085 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:07.418671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc4ad9-73e9-429f-8b46-aff76f1d8542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:07.419661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc4ad9-73e9-429f-8b46-aff76f1d8542 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:07.420187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc4ad9-73e9-429f-8b46-aff76f1d8542 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:07.420665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc4ad9-73e9-429f-8b46-aff76f1d8542 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:07.421566 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc4ad9-73e9-429f-8b46-aff76f1d8542 None admin] RBAC: Adding query filter params (name=network) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:07.422086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc4ad9-73e9-429f-8b46-aff76f1d8542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:07.423670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc4ad9-73e9-429f-8b46-aff76f1d8542 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:07.429228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 356/718] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:07.435427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2949f8-de64-492c-8e8a-f3e48c84d631 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:07.579277 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2949f8-de64-492c-8e8a-f3e48c84d631 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:07.605144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:07.621086 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ddf7a-2654-443b-ba4e-d724952aa3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:07.621941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ddf7a-2654-443b-ba4e-d724952aa3eb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:07.622225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ddf7a-2654-443b-ba4e-d724952aa3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:07.622502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ddf7a-2654-443b-ba4e-d724952aa3eb None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:07.623149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ddf7a-2654-443b-ba4e-d724952aa3eb None admin] RBAC: Adding query filter params (type=network) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:07.623454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ddf7a-2654-443b-ba4e-d724952aa3eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:07.624893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ddf7a-2654-443b-ba4e-d724952aa3eb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:07.630214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 363/719] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:07 2022] GET /identity/v3/services?type=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:07.945471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b99f-d01f-4b6d-9ca8-b9f6016bcc7e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:07.945761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b99f-d01f-4b6d-9ca8-b9f6016bcc7e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:07.946040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b99f-d01f-4b6d-9ca8-b9f6016bcc7e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:07.948184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 357/720] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:07.954516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2dfc8-9986-4207-a5c5-1ca29ef35adc None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:07.954804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2dfc8-9986-4207-a5c5-1ca29ef35adc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:07.955078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2dfc8-9986-4207-a5c5-1ca29ef35adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:07.979584 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-14c2dfc8-9986-4207-a5c5-1ca29ef35adc None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:07.981159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-14c2dfc8-9986-4207-a5c5-1ca29ef35adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:08.085464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 364/721] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:20:08.108874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749b99f-d01f-4b6d-9ca8-b9f6016bcc7e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:08.176488 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:08.246464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749b99f-d01f-4b6d-9ca8-b9f6016bcc7e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:08.291746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a0134-195e-4017-8c68-6dbd7cbc6b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:08.292413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a0134-195e-4017-8c68-6dbd7cbc6b57 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:08.292693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a0134-195e-4017-8c68-6dbd7cbc6b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:08.292970 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a0134-195e-4017-8c68-6dbd7cbc6b57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:08.309759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a0134-195e-4017-8c68-6dbd7cbc6b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d01a0134-195e-4017-8c68-6dbd7cbc6b57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:08.315076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:08.316168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:08.317028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 358/722] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:08.325246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2dfc8-9986-4207-a5c5-1ca29ef35adc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:08.500731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2dfc8-9986-4207-a5c5-1ca29ef35adc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:08.523022 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:08.546111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7853ca-9a50-4713-9266-2a40c29b506d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:08.546730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7853ca-9a50-4713-9266-2a40c29b506d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:08.547010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7853ca-9a50-4713-9266-2a40c29b506d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:08.547282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7853ca-9a50-4713-9266-2a40c29b506d None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:08.547954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7853ca-9a50-4713-9266-2a40c29b506d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:08.548285 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7853ca-9a50-4713-9266-2a40c29b506d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:08.549650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7853ca-9a50-4713-9266-2a40c29b506d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:08.564005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 365/723] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:20:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:08.573591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a0134-195e-4017-8c68-6dbd7cbc6b57 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:08.720268 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a0134-195e-4017-8c68-6dbd7cbc6b57 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:08.734014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:08.757034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4913a5-2e43-4042-a520-792c3f0ea6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:08.757586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4913a5-2e43-4042-a520-792c3f0ea6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:08.757865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4913a5-2e43-4042-a520-792c3f0ea6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:08.758186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4913a5-2e43-4042-a520-792c3f0ea6a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:08.775689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4913a5-2e43-4042-a520-792c3f0ea6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:08.777192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4913a5-2e43-4042-a520-792c3f0ea6a0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-5f4913a5-2e43-4042-a520-792c3f0ea6a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:08.796344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:08.797382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:08.798376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:08.798376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:08.798376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:08.798971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 359/724] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:08.803371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7853ca-9a50-4713-9266-2a40c29b506d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:08.946390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7853ca-9a50-4713-9266-2a40c29b506d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:09.004733 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b888c-8092-4909-8aa0-17e453205f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:09.005712 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b888c-8092-4909-8aa0-17e453205f5d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:09.006209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b888c-8092-4909-8aa0-17e453205f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:09.006687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b888c-8092-4909-8aa0-17e453205f5d None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:09.007602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b888c-8092-4909-8aa0-17e453205f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:09.008150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b888c-8092-4909-8aa0-17e453205f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:09.009748 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b888c-8092-4909-8aa0-17e453205f5d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:09.026589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 366/725] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:09.032326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4913a5-2e43-4042-a520-792c3f0ea6a0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:09.171804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4913a5-2e43-4042-a520-792c3f0ea6a0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:09.214288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbbae7-2dae-4c35-921f-30ef33cae5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:09.214924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbbae7-2dae-4c35-921f-30ef33cae5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:09.215208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbbae7-2dae-4c35-921f-30ef33cae5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:09.215484 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbbae7-2dae-4c35-921f-30ef33cae5e3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:09.227745 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cbbae7-2dae-4c35-921f-30ef33cae5e3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:09.230222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cbbae7-2dae-4c35-921f-30ef33cae5e3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-82cbbae7-2dae-4c35-921f-30ef33cae5e3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:09.245829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:09.246808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:09.247773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:09.247773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:09.247773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:09.247773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:09.247773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:09.247773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:09.247773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:09.247773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:09.247773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:20:09.247773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:09.247773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 360/726] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:09.252814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b888c-8092-4909-8aa0-17e453205f5d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:09.326521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:09.394475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b888c-8092-4909-8aa0-17e453205f5d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:09.430879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d016af-f097-438c-97fd-6b871aee8e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:09.431550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d016af-f097-438c-97fd-6b871aee8e18 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:09.431986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d016af-f097-438c-97fd-6b871aee8e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:09.432322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d016af-f097-438c-97fd-6b871aee8e18 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:09.433029 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d016af-f097-438c-97fd-6b871aee8e18 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:09.433339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d016af-f097-438c-97fd-6b871aee8e18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:09.434841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d016af-f097-438c-97fd-6b871aee8e18 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:09.455734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 367/727] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 22:20:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:09.461245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbbae7-2dae-4c35-921f-30ef33cae5e3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:09.508064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:09.545559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ef376-0ff4-4089-a3f6-599ae2e48ef1 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:09.545837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ef376-0ff4-4089-a3f6-599ae2e48ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:09.546112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ef376-0ff4-4089-a3f6-599ae2e48ef1 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:09.549793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 368/728] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:09.549793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d81f5-1377-4c71-91b8-dbd6bf5cbfd9 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:09.549793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d81f5-1377-4c71-91b8-dbd6bf5cbfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:09.549793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d81f5-1377-4c71-91b8-dbd6bf5cbfd9 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:09.550655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 369/729] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:09.556144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc1ae6-acfb-4784-8457-1466c238a00c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:09.556432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc1ae6-acfb-4784-8457-1466c238a00c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:09.557304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc1ae6-acfb-4784-8457-1466c238a00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:09.576182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-82cc1ae6-acfb-4784-8457-1466c238a00c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:09.577440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-82cc1ae6-acfb-4784-8457-1466c238a00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:09.604743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbbae7-2dae-4c35-921f-30ef33cae5e3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:09.644512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d02a5-3900-4926-b366-67db2970ae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:09.645122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d02a5-3900-4926-b366-67db2970ae36 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:09.645411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d02a5-3900-4926-b366-67db2970ae36 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:09.645687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d02a5-3900-4926-b366-67db2970ae36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:09.656161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 370/730] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:20:09.656161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa006ab-d0af-4ddc-ac33-4f3322b84dac None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:09.656161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa006ab-d0af-4ddc-ac33-4f3322b84dac None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:09.656161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa006ab-d0af-4ddc-ac33-4f3322b84dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:09.679684 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1fa006ab-d0af-4ddc-ac33-4f3322b84dac None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:09.682297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1fa006ab-d0af-4ddc-ac33-4f3322b84dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:09.708483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:09.719047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d02a5-3900-4926-b366-67db2970ae36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:09.720539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d02a5-3900-4926-b366-67db2970ae36 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:09.745746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:09.764314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 371/731] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:20:09.765622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ef865-8be4-47b1-992f-2f0754c7b356 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:09.765904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ef865-8be4-47b1-992f-2f0754c7b356 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:09.766179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ef865-8be4-47b1-992f-2f0754c7b356 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:09.767621 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 372/732] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:09.768437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ef865-8be4-47b1-992f-2f0754c7b356 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:09.834267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 361/733] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 22:20:09.835435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9e2e7-38cf-419f-897b-13243a7ca4ed None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:09.835720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9e2e7-38cf-419f-897b-13243a7ca4ed None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:09.835993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9e2e7-38cf-419f-897b-13243a7ca4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:09.855325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-29f9e2e7-38cf-419f-897b-13243a7ca4ed None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:09.856534 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-29f9e2e7-38cf-419f-897b-13243a7ca4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:09.889544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ef865-8be4-47b1-992f-2f0754c7b356 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:09.925293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864683-df79-4223-a566-fb5a7213982a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:09.925891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864683-df79-4223-a566-fb5a7213982a None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:09.926167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864683-df79-4223-a566-fb5a7213982a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:09.926437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864683-df79-4223-a566-fb5a7213982a None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:09.927074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16864683-df79-4223-a566-fb5a7213982a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:09.928461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16864683-df79-4223-a566-fb5a7213982a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:09.929329 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 362/734] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:20:09.930083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9e2e7-38cf-419f-897b-13243a7ca4ed None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:09.997809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:10.043680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9e2e7-38cf-419f-897b-13243a7ca4ed None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:10.055348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 373/735] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:20:09 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:10.055348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864683-df79-4223-a566-fb5a7213982a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:10.073896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:10.115277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bd938-9b75-43ea-b3b2-d20d3cd1d30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:10.116032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bd938-9b75-43ea-b3b2-d20d3cd1d30f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:10.116470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bd938-9b75-43ea-b3b2-d20d3cd1d30f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:10.117148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bd938-9b75-43ea-b3b2-d20d3cd1d30f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:10.132969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9bd938-9b75-43ea-b3b2-d20d3cd1d30f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-1b9bd938-9b75-43ea-b3b2-d20d3cd1d30f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:10.141228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:10.142225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:20:10.143530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:10.144222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 363/736] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:20:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:10.145031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bd938-9b75-43ea-b3b2-d20d3cd1d30f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:10.182029 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864683-df79-4223-a566-fb5a7213982a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:10.232996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83fbf1-bca7-4111-8224-746cf4344649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:10.233642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83fbf1-bca7-4111-8224-746cf4344649 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:10.233916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83fbf1-bca7-4111-8224-746cf4344649 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:10.234182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83fbf1-bca7-4111-8224-746cf4344649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:10.246326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83fbf1-bca7-4111-8224-746cf4344649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:10.247784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83fbf1-bca7-4111-8224-746cf4344649 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-fe83fbf1-bca7-4111-8224-746cf4344649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:10.265424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:10.266605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:10.267580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:10.267580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:10.267580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:10.267580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 374/737] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:10.267816 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83fbf1-bca7-4111-8224-746cf4344649 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:10.284961 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bd938-9b75-43ea-b3b2-d20d3cd1d30f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:10.297778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:10.325472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de421e3-06e5-4cba-9483-81ae22115652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:10.326098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de421e3-06e5-4cba-9483-81ae22115652 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:10.326377 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de421e3-06e5-4cba-9483-81ae22115652 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:10.326648 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de421e3-06e5-4cba-9483-81ae22115652 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:10.337661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de421e3-06e5-4cba-9483-81ae22115652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-6de421e3-06e5-4cba-9483-81ae22115652 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:10.343618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:10.344571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:10.345414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 364/738] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:10.346099 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de421e3-06e5-4cba-9483-81ae22115652 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:10.355712 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:10.396596 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83fbf1-bca7-4111-8224-746cf4344649 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:10.431407 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b107da-0a27-405f-9c77-53e642d86bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:10.432737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b107da-0a27-405f-9c77-53e642d86bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:10.433144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b107da-0a27-405f-9c77-53e642d86bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:10.433424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b107da-0a27-405f-9c77-53e642d86bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:10.434092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b107da-0a27-405f-9c77-53e642d86bd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:10.434398 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b107da-0a27-405f-9c77-53e642d86bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:10.435785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b107da-0a27-405f-9c77-53e642d86bd7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:10.448642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 375/739] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:20:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:10.448642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab63730-3a61-499f-aaa4-52a16b4db6a9 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:10.448642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab63730-3a61-499f-aaa4-52a16b4db6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:10.448642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab63730-3a61-499f-aaa4-52a16b4db6a9 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:10.448642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 376/740] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:10.449074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab63730-3a61-499f-aaa4-52a16b4db6a9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:10.460811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de421e3-06e5-4cba-9483-81ae22115652 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:10.499798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129b847-fd37-489e-a030-d065ec6cf2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:10.500895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129b847-fd37-489e-a030-d065ec6cf2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:10.501323 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129b847-fd37-489e-a030-d065ec6cf2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:10.501722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129b847-fd37-489e-a030-d065ec6cf2b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:10.502375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129b847-fd37-489e-a030-d065ec6cf2b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:10.502932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129b847-fd37-489e-a030-d065ec6cf2b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:10.504719 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129b847-fd37-489e-a030-d065ec6cf2b4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:10.523290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 365/741] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:10.524197 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129b847-fd37-489e-a030-d065ec6cf2b4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:10.552413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:10.562037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab63730-3a61-499f-aaa4-52a16b4db6a9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:10.592843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:10.598756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5a382-f806-41a0-905d-68121f3f0456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:10.599395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5a382-f806-41a0-905d-68121f3f0456 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:10.599680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5a382-f806-41a0-905d-68121f3f0456 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:10.599952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5a382-f806-41a0-905d-68121f3f0456 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:10.600645 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5a382-f806-41a0-905d-68121f3f0456 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:10.600951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5a382-f806-41a0-905d-68121f3f0456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:10.602319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5a382-f806-41a0-905d-68121f3f0456 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:10.611382 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 377/742] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:20:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:10.612440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12915098-3a7f-473c-8e8a-ee952e1572b3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:10.612723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12915098-3a7f-473c-8e8a-ee952e1572b3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:10.613000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12915098-3a7f-473c-8e8a-ee952e1572b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:10.631622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-12915098-3a7f-473c-8e8a-ee952e1572b3 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:10.632628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-12915098-3a7f-473c-8e8a-ee952e1572b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:10.646540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129b847-fd37-489e-a030-d065ec6cf2b4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:10.686680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc9b04-2d80-42cf-b1e3-61dc0e38e7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:10.687255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc9b04-2d80-42cf-b1e3-61dc0e38e7bf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:10.687536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc9b04-2d80-42cf-b1e3-61dc0e38e7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:10.687810 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc9b04-2d80-42cf-b1e3-61dc0e38e7bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:10.701674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc9b04-2d80-42cf-b1e3-61dc0e38e7bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:10.703096 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc9b04-2d80-42cf-b1e3-61dc0e38e7bf None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:10.703614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 378/743] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:10 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 22:20:10.704386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12915098-3a7f-473c-8e8a-ee952e1572b3 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-3bfc9b04-2d80-42cf-b1e3-61dc0e38e7bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:10.729293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:10.730280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:10.731158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:10.731158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:10.731158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:10.731158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 366/744] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:10.731158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc9b04-2d80-42cf-b1e3-61dc0e38e7bf None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:10.815851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:10.824705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12915098-3a7f-473c-8e8a-ee952e1572b3 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:10.838796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc9b04-2d80-42cf-b1e3-61dc0e38e7bf None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:10.855544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:10.866565 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fb6c9-883c-43d0-8182-4857f08d67e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:10.869909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fb6c9-883c-43d0-8182-4857f08d67e7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:10.869909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fb6c9-883c-43d0-8182-4857f08d67e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:10.869909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fb6c9-883c-43d0-8182-4857f08d67e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:10.875333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76037f-1f52-4554-9778-c9a8bb5e1a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:10.875786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fb6c9-883c-43d0-8182-4857f08d67e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:10.876148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76037f-1f52-4554-9778-c9a8bb5e1a9d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:10.876497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76037f-1f52-4554-9778-c9a8bb5e1a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:10.876833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76037f-1f52-4554-9778-c9a8bb5e1a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:10.877689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fb6c9-883c-43d0-8182-4857f08d67e7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:10.888062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76037f-1f52-4554-9778-c9a8bb5e1a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:10.889561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76037f-1f52-4554-9778-c9a8bb5e1a9d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-123fb6c9-883c-43d0-8182-4857f08d67e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:10.890797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:10.891836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:10.893004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:10.893004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:10.893004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:10.893004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:10.893004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:10.893004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:10.893004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:10.893004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:10.893682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 379/745] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:10.898560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fb6c9-883c-43d0-8182-4857f08d67e7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-fc76037f-1f52-4554-9778-c9a8bb5e1a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:10.913486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:10.914632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:10.915804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:10.915804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:10.915804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:10.919468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 367/746] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:10.920309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76037f-1f52-4554-9778-c9a8bb5e1a9d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:11.011562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fb6c9-883c-43d0-8182-4857f08d67e7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:11.033104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76037f-1f52-4554-9778-c9a8bb5e1a9d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:11.047932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12814d04-9ba5-46e5-87e7-8811d7a6332f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:11.048840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12814d04-9ba5-46e5-87e7-8811d7a6332f None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:11.049124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12814d04-9ba5-46e5-87e7-8811d7a6332f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:11.049397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12814d04-9ba5-46e5-87e7-8811d7a6332f None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:11.050193 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12814d04-9ba5-46e5-87e7-8811d7a6332f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:11.051888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12814d04-9ba5-46e5-87e7-8811d7a6332f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:11.062904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:11.067892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1855a2-0d0d-450e-a312-302fa7a608b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:11.072832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1855a2-0d0d-450e-a312-302fa7a608b6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:11.072832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1855a2-0d0d-450e-a312-302fa7a608b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:11.072832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1855a2-0d0d-450e-a312-302fa7a608b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:11.072832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1855a2-0d0d-450e-a312-302fa7a608b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:11.072832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1855a2-0d0d-450e-a312-302fa7a608b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:11.072832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1855a2-0d0d-450e-a312-302fa7a608b6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:11.082335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 368/747] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:11.083116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1855a2-0d0d-450e-a312-302fa7a608b6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:11.160053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 380/748] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:20:10 2022] POST /identity/v3/services => generated 240 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:11.160053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12814d04-9ba5-46e5-87e7-8811d7a6332f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:11.199585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1855a2-0d0d-450e-a312-302fa7a608b6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:11.208844 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:11.262239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cfe0e-d50c-49ad-8f5a-2f92c8a9d938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:11.263137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cfe0e-d50c-49ad-8f5a-2f92c8a9d938 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:11.263415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cfe0e-d50c-49ad-8f5a-2f92c8a9d938 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:11.263686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cfe0e-d50c-49ad-8f5a-2f92c8a9d938 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:11.264587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cfe0e-d50c-49ad-8f5a-2f92c8a9d938 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:11.265206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cfe0e-d50c-49ad-8f5a-2f92c8a9d938 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:11.266715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cfe0e-d50c-49ad-8f5a-2f92c8a9d938 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:11.283270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12814d04-9ba5-46e5-87e7-8811d7a6332f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:11.286913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 369/749] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:11.287789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cfe0e-d50c-49ad-8f5a-2f92c8a9d938 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:11.323260 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa9931-d04d-4445-85be-de11676ca9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:11.323893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9931-d04d-4445-85be-de11676ca9df None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:11.324217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9931-d04d-4445-85be-de11676ca9df None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:11.324491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9931-d04d-4445-85be-de11676ca9df None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:11.325208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa9931-d04d-4445-85be-de11676ca9df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:11.325517 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa9931-d04d-4445-85be-de11676ca9df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:11.326963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa9931-d04d-4445-85be-de11676ca9df None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:11.340135 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 381/750] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:11.341922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa9931-d04d-4445-85be-de11676ca9df None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:11.350332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:11.417444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cfe0e-d50c-49ad-8f5a-2f92c8a9d938 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:11.459561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa9931-d04d-4445-85be-de11676ca9df None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:11.463687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851be4d-46c8-4d5a-9a5e-ec0f7a39b7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:11.464587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851be4d-46c8-4d5a-9a5e-ec0f7a39b7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:11.465008 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851be4d-46c8-4d5a-9a5e-ec0f7a39b7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:11.465420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851be4d-46c8-4d5a-9a5e-ec0f7a39b7f5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:11.472507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:11.475001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851be4d-46c8-4d5a-9a5e-ec0f7a39b7f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:11.477209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851be4d-46c8-4d5a-9a5e-ec0f7a39b7f5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c851be4d-46c8-4d5a-9a5e-ec0f7a39b7f5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:11.493179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:11.494953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:11.496381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:11.496381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:11.496381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:11.496381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:11.496381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:11.496381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:11.496381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:11.496381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:11.496381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:11.496381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:11.497401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 370/751] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 22:20:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:11.502343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851be4d-46c8-4d5a-9a5e-ec0f7a39b7f5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:11.502552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64293e6-aca0-4c8f-b6fa-aeb3a5edb7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:11.502552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64293e6-aca0-4c8f-b6fa-aeb3a5edb7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:11.502552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64293e6-aca0-4c8f-b6fa-aeb3a5edb7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:11.502552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64293e6-aca0-4c8f-b6fa-aeb3a5edb7c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:11.503167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64293e6-aca0-4c8f-b6fa-aeb3a5edb7c2 None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=e86991778c2b47c58f10d9d9e3ca5e82) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:11.503311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64293e6-aca0-4c8f-b6fa-aeb3a5edb7c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:11.504773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64293e6-aca0-4c8f-b6fa-aeb3a5edb7c2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:11.516103 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 382/752] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:11 2022] GET /identity/v3/role_assignments?user.id=e86991778c2b47c58f10d9d9e3ca5e82&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=108664305d2546949f7327b65d4cb667 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:11.517629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64293e6-aca0-4c8f-b6fa-aeb3a5edb7c2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:11.651993 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64293e6-aca0-4c8f-b6fa-aeb3a5edb7c2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:11.671517 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851be4d-46c8-4d5a-9a5e-ec0f7a39b7f5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:11.690622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:11.708751 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68a6a0-20ac-4e3b-a0b7-f257397bf059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:11.709382 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68a6a0-20ac-4e3b-a0b7-f257397bf059 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:11.709713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68a6a0-20ac-4e3b-a0b7-f257397bf059 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:11.709926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68a6a0-20ac-4e3b-a0b7-f257397bf059 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:11.710704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68a6a0-20ac-4e3b-a0b7-f257397bf059 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:11.711006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68a6a0-20ac-4e3b-a0b7-f257397bf059 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:11.712531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68a6a0-20ac-4e3b-a0b7-f257397bf059 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:11.731594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0d861-769c-41ca-98a9-ab44816d1359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:11.732326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0d861-769c-41ca-98a9-ab44816d1359 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:11.732614 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0d861-769c-41ca-98a9-ab44816d1359 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:11.732883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0d861-769c-41ca-98a9-ab44816d1359 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:11.748658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 383/753] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 22:20:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:11.754307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68a6a0-20ac-4e3b-a0b7-f257397bf059 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:11.755426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0d861-769c-41ca-98a9-ab44816d1359 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:11.756918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0d861-769c-41ca-98a9-ab44816d1359 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ccf0d861-769c-41ca-98a9-ab44816d1359 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:11.782311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:11.783325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:11.784334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:11.784334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:11.784334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:11.784334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:11.784334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:11.784334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:11.784334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:11.784334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:11.784334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:11.784334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:11.784334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 371/754] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 22:20:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:11.793087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0d861-769c-41ca-98a9-ab44816d1359 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:11.828397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:11.910419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0d861-769c-41ca-98a9-ab44816d1359 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:11.955236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010eadf-1c6a-4423-b4af-502720e38335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:11.956299 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010eadf-1c6a-4423-b4af-502720e38335 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:11.956817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010eadf-1c6a-4423-b4af-502720e38335 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:11.957306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010eadf-1c6a-4423-b4af-502720e38335 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:11.958380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010eadf-1c6a-4423-b4af-502720e38335 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:11.958891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010eadf-1c6a-4423-b4af-502720e38335 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:11.960621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010eadf-1c6a-4423-b4af-502720e38335 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:11.985335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 372/755] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 22:20:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:11.991078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010eadf-1c6a-4423-b4af-502720e38335 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:12.039222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:12.061593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68a6a0-20ac-4e3b-a0b7-f257397bf059 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:12.098380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baec6e2-5952-4337-9e52-f70ac5ff5888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:12.098998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baec6e2-5952-4337-9e52-f70ac5ff5888 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:12.099276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baec6e2-5952-4337-9e52-f70ac5ff5888 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:12.099551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baec6e2-5952-4337-9e52-f70ac5ff5888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:12.158537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010eadf-1c6a-4423-b4af-502720e38335 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:12.179324 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baec6e2-5952-4337-9e52-f70ac5ff5888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:12.181115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baec6e2-5952-4337-9e52-f70ac5ff5888 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:12.209078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:12.217753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b767f1-ea23-4c38-a901-cac44fe34512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:12.217753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b767f1-ea23-4c38-a901-cac44fe34512 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:12.217753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b767f1-ea23-4c38-a901-cac44fe34512 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:12.217753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b767f1-ea23-4c38-a901-cac44fe34512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:12.246062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b767f1-ea23-4c38-a901-cac44fe34512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:12.247540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b767f1-ea23-4c38-a901-cac44fe34512 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-56b767f1-ea23-4c38-a901-cac44fe34512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:12.279196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:12.280312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:12.281176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:12.281176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:12.281176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:12.287979 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 373/756] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:12.288872 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b767f1-ea23-4c38-a901-cac44fe34512 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:12.290448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 384/757] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 22:20:12.296952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baec6e2-5952-4337-9e52-f70ac5ff5888 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:12.394003 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:12.409581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baec6e2-5952-4337-9e52-f70ac5ff5888 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:12.421712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b767f1-ea23-4c38-a901-cac44fe34512 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:12.435203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:12.446799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c2ad6-f826-45e9-8a19-f8f0af111a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:12.448314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c2ad6-f826-45e9-8a19-f8f0af111a13 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:12.448818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c2ad6-f826-45e9-8a19-f8f0af111a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:12.449243 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c2ad6-f826-45e9-8a19-f8f0af111a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:12.460624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c2ad6-f826-45e9-8a19-f8f0af111a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:12.462135 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c2ad6-f826-45e9-8a19-f8f0af111a13 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:12.465362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2f491-a69e-446b-9afa-4f951ff15075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:12.466131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2f491-a69e-446b-9afa-4f951ff15075 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:12.466942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2f491-a69e-446b-9afa-4f951ff15075 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:12.467225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2f491-a69e-446b-9afa-4f951ff15075 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:12.467927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2f491-a69e-446b-9afa-4f951ff15075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:12.468209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2f491-a69e-446b-9afa-4f951ff15075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:12.469549 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2f491-a69e-446b-9afa-4f951ff15075 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-409c2ad6-f826-45e9-8a19-f8f0af111a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:12.480670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:12.481602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:12.482486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:12.482486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:12.482486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:12.483501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 374/758] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:12.488050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f4ad3-e3a0-4f0f-aab6-1eacdf34cdce None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:12.488050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f4ad3-e3a0-4f0f-aab6-1eacdf34cdce None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:12.488050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f4ad3-e3a0-4f0f-aab6-1eacdf34cdce None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:12.488050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 375/759] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:12.488373 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f4ad3-e3a0-4f0f-aab6-1eacdf34cdce None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:12.488695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 385/760] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:12.492005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c2ad6-f826-45e9-8a19-f8f0af111a13 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:12.602425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c2ad6-f826-45e9-8a19-f8f0af111a13 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:12.624555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f4ad3-e3a0-4f0f-aab6-1eacdf34cdce None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:12.645513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371cfd8-2d75-4989-bb45-d1ac96dc88d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:12.646172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371cfd8-2d75-4989-bb45-d1ac96dc88d4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:12.646455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371cfd8-2d75-4989-bb45-d1ac96dc88d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:12.646728 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371cfd8-2d75-4989-bb45-d1ac96dc88d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:12.647887 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371cfd8-2d75-4989-bb45-d1ac96dc88d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:12.648233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371cfd8-2d75-4989-bb45-d1ac96dc88d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:12.650066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371cfd8-2d75-4989-bb45-d1ac96dc88d4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:12.663646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85514de1-0214-4c3f-ae58-85c5bcac4a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:12.664654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514de1-0214-4c3f-ae58-85c5bcac4a41 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:12.665132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514de1-0214-4c3f-ae58-85c5bcac4a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:12.665599 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514de1-0214-4c3f-ae58-85c5bcac4a41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:12.670577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 386/761] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:12.670577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f844a-ee02-4b4b-b91a-85a8fabffecc None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:12.670577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f844a-ee02-4b4b-b91a-85a8fabffecc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:12.670577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f844a-ee02-4b4b-b91a-85a8fabffecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:12.678158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85514de1-0214-4c3f-ae58-85c5bcac4a41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:12.679701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85514de1-0214-4c3f-ae58-85c5bcac4a41 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:12.682170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-85514de1-0214-4c3f-ae58-85c5bcac4a41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:12.692272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:12.693432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:12.694333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:12.694333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:12.694333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:12.694333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:12.694333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:12.694333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:12.694333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:12.694333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:12.694333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 376/762] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:12.694850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85514de1-0214-4c3f-ae58-85c5bcac4a41 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:12.719550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f54f844a-ee02-4b4b-b91a-85a8fabffecc None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:12.720798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f54f844a-ee02-4b4b-b91a-85a8fabffecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:12.764166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:12.810177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 387/763] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:12.811030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f844a-ee02-4b4b-b91a-85a8fabffecc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:12.812764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85514de1-0214-4c3f-ae58-85c5bcac4a41 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:12.852150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8fe1-ab8a-44ff-ba16-1a99dac88739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:12.853133 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8fe1-ab8a-44ff-ba16-1a99dac88739 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:12.853800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8fe1-ab8a-44ff-ba16-1a99dac88739 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:12.854304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8fe1-ab8a-44ff-ba16-1a99dac88739 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:12.862139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd8fe1-ab8a-44ff-ba16-1a99dac88739 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:12.863970 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd8fe1-ab8a-44ff-ba16-1a99dac88739 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-dbfd8fe1-ab8a-44ff-ba16-1a99dac88739 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:12.877569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:12.878490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:12.879390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:12.879390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:12.879390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:12.879390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:12.879390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:12.879390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:12.879390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:12.879390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:12.881836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 377/764] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:12.882961 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8fe1-ab8a-44ff-ba16-1a99dac88739 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:12.923576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f844a-ee02-4b4b-b91a-85a8fabffecc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:12.966984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a54c4c-b75f-430d-b1bf-2eac64dacf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:12.967977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a54c4c-b75f-430d-b1bf-2eac64dacf98 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:12.968732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a54c4c-b75f-430d-b1bf-2eac64dacf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:12.969427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a54c4c-b75f-430d-b1bf-2eac64dacf98 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:12.970337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a54c4c-b75f-430d-b1bf-2eac64dacf98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:12.970733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:12.970938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a54c4c-b75f-430d-b1bf-2eac64dacf98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:12.972989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a54c4c-b75f-430d-b1bf-2eac64dacf98 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:12.990560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 388/765] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:12.992177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a54c4c-b75f-430d-b1bf-2eac64dacf98 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:13.017829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8fe1-ab8a-44ff-ba16-1a99dac88739 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:13.029108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:13.054556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86df0be-93b6-4166-bd55-5b36a5d288aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:13.055355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df0be-93b6-4166-bd55-5b36a5d288aa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:13.055629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df0be-93b6-4166-bd55-5b36a5d288aa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:13.055971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df0be-93b6-4166-bd55-5b36a5d288aa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:13.056764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86df0be-93b6-4166-bd55-5b36a5d288aa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:13.058451 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86df0be-93b6-4166-bd55-5b36a5d288aa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d86df0be-93b6-4166-bd55-5b36a5d288aa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:13.068442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:13.069502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 22:20:13.070340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:13.075437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 378/766] 10.222.0.30 () {56 vars in 1129 bytes} [Sun Aug 7 22:20:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:13.076518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86df0be-93b6-4166-bd55-5b36a5d288aa None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:13.124241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a54c4c-b75f-430d-b1bf-2eac64dacf98 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:13.163853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71a8ce-ecce-4a80-9a47-86225f5445eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:13.164609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71a8ce-ecce-4a80-9a47-86225f5445eb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:13.164954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71a8ce-ecce-4a80-9a47-86225f5445eb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:13.165298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71a8ce-ecce-4a80-9a47-86225f5445eb None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:13.166577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71a8ce-ecce-4a80-9a47-86225f5445eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:13.166946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71a8ce-ecce-4a80-9a47-86225f5445eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:13.168582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71a8ce-ecce-4a80-9a47-86225f5445eb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:13.188066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 389/767] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:13.188066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71a8ce-ecce-4a80-9a47-86225f5445eb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:13.204715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86df0be-93b6-4166-bd55-5b36a5d288aa None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:13.233329 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:13.248200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c7dd8-2cfd-497f-b34a-926fb3cc2d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:13.249346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c7dd8-2cfd-497f-b34a-926fb3cc2d06 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:13.249805 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c7dd8-2cfd-497f-b34a-926fb3cc2d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:13.250400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c7dd8-2cfd-497f-b34a-926fb3cc2d06 None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/8effbd0670ef41f59b46feb9f0997554/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:13.254074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c7dd8-2cfd-497f-b34a-926fb3cc2d06 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=8effbd0670ef41f59b46feb9f0997554, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:13.255562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c7dd8-2cfd-497f-b34a-926fb3cc2d06 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:13.285015 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:13.299525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71a8ce-ecce-4a80-9a47-86225f5445eb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:13.334053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de325e67-d80b-49f5-af8c-abb00abcce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:13.334659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325e67-d80b-49f5-af8c-abb00abcce1f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:13.334932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325e67-d80b-49f5-af8c-abb00abcce1f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:13.335224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325e67-d80b-49f5-af8c-abb00abcce1f None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:13.335830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de325e67-d80b-49f5-af8c-abb00abcce1f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:13.336163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de325e67-d80b-49f5-af8c-abb00abcce1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:13.337518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de325e67-d80b-49f5-af8c-abb00abcce1f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:13.346465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 390/768] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 22:20:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:13.346465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de325e67-d80b-49f5-af8c-abb00abcce1f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:13.364638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 379/769] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:20:13 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/8effbd0670ef41f59b46feb9f0997554/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:20:13.370595 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c7dd8-2cfd-497f-b34a-926fb3cc2d06 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:13.522617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de325e67-d80b-49f5-af8c-abb00abcce1f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:13.557574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c7dd8-2cfd-497f-b34a-926fb3cc2d06 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:13.569699 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4c4b7-3e9a-4ffa-8dc3-887733dd278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:13.570399 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c4b7-3e9a-4ffa-8dc3-887733dd278a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:13.570597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c4b7-3e9a-4ffa-8dc3-887733dd278a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:13.570861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c4b7-3e9a-4ffa-8dc3-887733dd278a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:13.572955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4c4b7-3e9a-4ffa-8dc3-887733dd278a None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=77cb38c8c0dc4de3bbdeb611ffd53ff3) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:13.572955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4c4b7-3e9a-4ffa-8dc3-887733dd278a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:13.574867 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4c4b7-3e9a-4ffa-8dc3-887733dd278a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:13.581665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 391/770] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:13 2022] GET /identity/v3/role_assignments?user.id=77cb38c8c0dc4de3bbdeb611ffd53ff3&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:13.582987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154aa25-21c2-4772-a826-3dd3e9250137 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:13.583277 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154aa25-21c2-4772-a826-3dd3e9250137 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:13.583557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154aa25-21c2-4772-a826-3dd3e9250137 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:13.588120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 392/771] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:13.591618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fed0d9-5621-4510-aea7-fb6567ace7c3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:13.591910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fed0d9-5621-4510-aea7-fb6567ace7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:13.592212 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fed0d9-5621-4510-aea7-fb6567ace7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:13.623804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f942c-839b-441c-a4bc-c2871089d100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:13.624880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f942c-839b-441c-a4bc-c2871089d100 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:13.625420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f942c-839b-441c-a4bc-c2871089d100 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:13.625954 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f942c-839b-441c-a4bc-c2871089d100 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:13.626808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f942c-839b-441c-a4bc-c2871089d100 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:13.627316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f942c-839b-441c-a4bc-c2871089d100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:13.629972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f942c-839b-441c-a4bc-c2871089d100 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:13.638965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a0fed0d9-5621-4510-aea7-fb6567ace7c3 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:13.641819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a0fed0d9-5621-4510-aea7-fb6567ace7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:13.647649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 380/772] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 22:20:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:13.653834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f942c-839b-441c-a4bc-c2871089d100 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:13.670011 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:13.760659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:13.792213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 393/773] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:13.807939 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b530-75e2-4081-abec-20eaea8a2e7d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:13.808558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b530-75e2-4081-abec-20eaea8a2e7d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:13.809000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b530-75e2-4081-abec-20eaea8a2e7d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:13.810804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 394/774] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:13.817542 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96481129-c21f-480e-b5b5-785412a9e25e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:13.818084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96481129-c21f-480e-b5b5-785412a9e25e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:13.818550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96481129-c21f-480e-b5b5-785412a9e25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:13.839447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-96481129-c21f-480e-b5b5-785412a9e25e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:13.841020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-96481129-c21f-480e-b5b5-785412a9e25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:13.849481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f942c-839b-441c-a4bc-c2871089d100 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:13.885826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e759a43-63c7-4f41-9fc8-c0714d774dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:13.886443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e759a43-63c7-4f41-9fc8-c0714d774dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:13.886730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e759a43-63c7-4f41-9fc8-c0714d774dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:13.887006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e759a43-63c7-4f41-9fc8-c0714d774dd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:13.888047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e759a43-63c7-4f41-9fc8-c0714d774dd5 None admin] RBAC: Adding query filter params (interface=public, service_id=0a765e6fd86343b3a00ecb0e4d937962, region_id=RegionOne) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:13.888158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e759a43-63c7-4f41-9fc8-c0714d774dd5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:13.889775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e759a43-63c7-4f41-9fc8-c0714d774dd5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:13.897984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 381/775] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 22:20:13 2022] GET /identity/v3/endpoints?service_id=0a765e6fd86343b3a00ecb0e4d937962&interface=public®ion_id=RegionOne => generated 194 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:13.903241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e759a43-63c7-4f41-9fc8-c0714d774dd5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:13.924133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 395/776] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:13.925370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96481129-c21f-480e-b5b5-785412a9e25e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:14.020203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e759a43-63c7-4f41-9fc8-c0714d774dd5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:14.035091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:14.041259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:14.065159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235be609-75b4-4e2c-974c-4b78ca731bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:14.065722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be609-75b4-4e2c-974c-4b78ca731bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:14.066003 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be609-75b4-4e2c-974c-4b78ca731bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:14.066614 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be609-75b4-4e2c-974c-4b78ca731bb6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:14.067227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235be609-75b4-4e2c-974c-4b78ca731bb6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:14.069525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235be609-75b4-4e2c-974c-4b78ca731bb6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:14.074494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96481129-c21f-480e-b5b5-785412a9e25e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-235be609-75b4-4e2c-974c-4b78ca731bb6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:14.081272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:14.082478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 22:20:14.083494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:14.087715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 382/777] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:13 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:14.089366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235be609-75b4-4e2c-974c-4b78ca731bb6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:14.130094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f506f96-821d-433f-8d8a-1542c9e34419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:14.131225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f506f96-821d-433f-8d8a-1542c9e34419 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:14.131647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f506f96-821d-433f-8d8a-1542c9e34419 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:14.132304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f506f96-821d-433f-8d8a-1542c9e34419 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:14.133460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f506f96-821d-433f-8d8a-1542c9e34419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:14.135591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f506f96-821d-433f-8d8a-1542c9e34419 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:14.142799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 396/778] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 22:20:13 2022] GET /identity/v3/services => generated 772 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:14.144329 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f506f96-821d-433f-8d8a-1542c9e34419 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:14.270642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f506f96-821d-433f-8d8a-1542c9e34419 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:14.314179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235be609-75b4-4e2c-974c-4b78ca731bb6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:14.315051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053957a-c0fe-46a3-875e-de322e5fee41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:14.316202 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053957a-c0fe-46a3-875e-de322e5fee41 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:14.316652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053957a-c0fe-46a3-875e-de322e5fee41 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:14.317106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053957a-c0fe-46a3-875e-de322e5fee41 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:14.317912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053957a-c0fe-46a3-875e-de322e5fee41 None admin] RBAC: Adding query filter params (name=network) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:14.318379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053957a-c0fe-46a3-875e-de322e5fee41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:14.319919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053957a-c0fe-46a3-875e-de322e5fee41 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:14.326642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 397/779] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:14.333050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053957a-c0fe-46a3-875e-de322e5fee41 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:14.336840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:14.398780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a09d31-a7bd-48bf-b06b-567bae4eabc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:14.399513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a09d31-a7bd-48bf-b06b-567bae4eabc0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:14.399884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a09d31-a7bd-48bf-b06b-567bae4eabc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:14.400254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a09d31-a7bd-48bf-b06b-567bae4eabc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:14.415601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a09d31-a7bd-48bf-b06b-567bae4eabc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-e1a09d31-a7bd-48bf-b06b-567bae4eabc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:14.421733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:14.422670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:14.423494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 383/780] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:14.432228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a09d31-a7bd-48bf-b06b-567bae4eabc0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:14.467188 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053957a-c0fe-46a3-875e-de322e5fee41 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:14.474112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:14.506699 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362afb7-78df-47d2-b77b-6167db554610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:14.507363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362afb7-78df-47d2-b77b-6167db554610 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:14.507649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362afb7-78df-47d2-b77b-6167db554610 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:14.507922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362afb7-78df-47d2-b77b-6167db554610 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:14.508590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362afb7-78df-47d2-b77b-6167db554610 None admin] RBAC: Adding query filter params (type=network) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:14.508893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362afb7-78df-47d2-b77b-6167db554610 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:14.510777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362afb7-78df-47d2-b77b-6167db554610 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:14.520050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 398/781] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:14.522292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362afb7-78df-47d2-b77b-6167db554610 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:14.585193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a09d31-a7bd-48bf-b06b-567bae4eabc0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:14.597872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:14.637353 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362afb7-78df-47d2-b77b-6167db554610 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:14.672140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633e14a-cffb-4963-a1c2-310fb138efc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:14.672885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633e14a-cffb-4963-a1c2-310fb138efc8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:14.673178 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633e14a-cffb-4963-a1c2-310fb138efc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:14.673522 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633e14a-cffb-4963-a1c2-310fb138efc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:14.674085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1e422-622a-4ae6-9b2f-75bc01bea080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:14.674205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633e14a-cffb-4963-a1c2-310fb138efc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:14.674490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633e14a-cffb-4963-a1c2-310fb138efc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:14.675248 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1e422-622a-4ae6-9b2f-75bc01bea080 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:14.675717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1e422-622a-4ae6-9b2f-75bc01bea080 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:14.675900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633e14a-cffb-4963-a1c2-310fb138efc8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:14.676378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1e422-622a-4ae6-9b2f-75bc01bea080 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:14.677282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1e422-622a-4ae6-9b2f-75bc01bea080 None admin] RBAC: Adding query filter params (interface=public, service_id=67e682594337428b84f3b638b074ccb1, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:14.677861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1e422-622a-4ae6-9b2f-75bc01bea080 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:14.679426 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1e422-622a-4ae6-9b2f-75bc01bea080 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:14.687935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 399/782] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 22:20:14 2022] GET /identity/v3/endpoints?service_id=67e682594337428b84f3b638b074ccb1&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:14.693549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1e422-622a-4ae6-9b2f-75bc01bea080 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:14.698596 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 384/783] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:20:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:14.704129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633e14a-cffb-4963-a1c2-310fb138efc8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:14.803545 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1e422-622a-4ae6-9b2f-75bc01bea080 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:14.834437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:14.846036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe244135-996c-4190-bbfc-cdcb9d5991a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:14.846976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244135-996c-4190-bbfc-cdcb9d5991a0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:14.847521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244135-996c-4190-bbfc-cdcb9d5991a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:14.848684 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244135-996c-4190-bbfc-cdcb9d5991a0 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:14.849461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe244135-996c-4190-bbfc-cdcb9d5991a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:14.850986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe244135-996c-4190-bbfc-cdcb9d5991a0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:14.862158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 400/784] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 22:20:14 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:14.920425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:14.932943 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633e14a-cffb-4963-a1c2-310fb138efc8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:14.995893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5de65-d9f8-42f8-9e47-74d671745f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:14.996563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5de65-d9f8-42f8-9e47-74d671745f02 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:14.996896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5de65-d9f8-42f8-9e47-74d671745f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:14.997181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5de65-d9f8-42f8-9e47-74d671745f02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:15.014316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5de65-d9f8-42f8-9e47-74d671745f02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:15.015786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5de65-d9f8-42f8-9e47-74d671745f02 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-93a5de65-d9f8-42f8-9e47-74d671745f02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:15.033865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:15.035105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:15.036248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:15.036248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:15.036248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:15.039695 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 385/785] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:15.041211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe244135-996c-4190-bbfc-cdcb9d5991a0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:15.154014 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe244135-996c-4190-bbfc-cdcb9d5991a0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:15.190137 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31c734-4076-4e49-a265-6c3bf3c34662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:15.190745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31c734-4076-4e49-a265-6c3bf3c34662 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:15.191018 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31c734-4076-4e49-a265-6c3bf3c34662 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:15.191287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31c734-4076-4e49-a265-6c3bf3c34662 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:15.191934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31c734-4076-4e49-a265-6c3bf3c34662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:15.192286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31c734-4076-4e49-a265-6c3bf3c34662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:15.193648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31c734-4076-4e49-a265-6c3bf3c34662 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:15.207125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 401/786] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:15.217110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5de65-d9f8-42f8-9e47-74d671745f02 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:15.353664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5de65-d9f8-42f8-9e47-74d671745f02 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:15.403995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586b148-fe4c-4efb-89a9-420c88d3b177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:15.405083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b148-fe4c-4efb-89a9-420c88d3b177 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:15.405564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b148-fe4c-4efb-89a9-420c88d3b177 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:15.406034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b148-fe4c-4efb-89a9-420c88d3b177 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:15.413879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586b148-fe4c-4efb-89a9-420c88d3b177 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:15.415563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586b148-fe4c-4efb-89a9-420c88d3b177 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-7586b148-fe4c-4efb-89a9-420c88d3b177 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:15.427981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:15.428949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:15.429888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:15.429888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:15.429888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:15.429888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:15.429888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:15.429888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:15.429888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:15.429888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:15.429888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:20:15.429888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:15.436113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 386/787] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:15.438901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31c734-4076-4e49-a265-6c3bf3c34662 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:15.544133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:15.590457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31c734-4076-4e49-a265-6c3bf3c34662 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:15.635346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec8f6f-b1a8-4f6a-9d2f-e0612c96c0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:15.635972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec8f6f-b1a8-4f6a-9d2f-e0612c96c0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:15.636267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec8f6f-b1a8-4f6a-9d2f-e0612c96c0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:15.636540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec8f6f-b1a8-4f6a-9d2f-e0612c96c0f8 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:15.637274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec8f6f-b1a8-4f6a-9d2f-e0612c96c0f8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:15.637575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec8f6f-b1a8-4f6a-9d2f-e0612c96c0f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:15.639057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec8f6f-b1a8-4f6a-9d2f-e0612c96c0f8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:15.661770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 402/788] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 22:20:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:15.673125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586b148-fe4c-4efb-89a9-420c88d3b177 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:15.697707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:15.746294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcb12a-f698-4eb4-a930-23a1d3291119 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:15.746628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcb12a-f698-4eb4-a930-23a1d3291119 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:15.746918 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcb12a-f698-4eb4-a930-23a1d3291119 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:15.748439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 403/789] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:20:15.755158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec721c5-fb5a-4c31-b40b-fa321b9f2278 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:15.755438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec721c5-fb5a-4c31-b40b-fa321b9f2278 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:15.755709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec721c5-fb5a-4c31-b40b-fa321b9f2278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:15.791900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1ec721c5-fb5a-4c31-b40b-fa321b9f2278 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:15.795112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1ec721c5-fb5a-4c31-b40b-fa321b9f2278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:15.843835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586b148-fe4c-4efb-89a9-420c88d3b177 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:15.874146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 404/790] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:15.898710 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedbb15-6a8f-4fe2-a18d-4441989d2b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:15.899306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedbb15-6a8f-4fe2-a18d-4441989d2b5a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:15.899583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedbb15-6a8f-4fe2-a18d-4441989d2b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:15.899854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedbb15-6a8f-4fe2-a18d-4441989d2b5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:15.901687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec721c5-fb5a-4c31-b40b-fa321b9f2278 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:15.916116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedbb15-6a8f-4fe2-a18d-4441989d2b5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:15.916116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedbb15-6a8f-4fe2-a18d-4441989d2b5a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-2cedbb15-6a8f-4fe2-a18d-4441989d2b5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:15.938110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:15.939505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:15.940891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:15.940891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:15.940891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:15.940891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 387/791] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:15.940891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973b44-f5c9-435a-98ac-605ace2b21ba None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:15.941182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973b44-f5c9-435a-98ac-605ace2b21ba None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:15.941429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973b44-f5c9-435a-98ac-605ace2b21ba None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:15.942860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 388/792] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:15.949543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa973b44-f5c9-435a-98ac-605ace2b21ba None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:15.981337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:16.020763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:16.056457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec721c5-fb5a-4c31-b40b-fa321b9f2278 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:16.074500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa973b44-f5c9-435a-98ac-605ace2b21ba None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:16.093416 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af535110-6230-4ad5-b338-71434a2b775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:16.094251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af535110-6230-4ad5-b338-71434a2b775c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.094675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af535110-6230-4ad5-b338-71434a2b775c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.095118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af535110-6230-4ad5-b338-71434a2b775c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.107032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af535110-6230-4ad5-b338-71434a2b775c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-af535110-6230-4ad5-b338-71434a2b775c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:16.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:16.113724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:20:16.114708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:16.116279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd045697-1223-445b-8123-91fc8aa77cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:16.116885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd045697-1223-445b-8123-91fc8aa77cfa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.117201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd045697-1223-445b-8123-91fc8aa77cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.117338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 405/793] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:20:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:16.117419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd045697-1223-445b-8123-91fc8aa77cfa None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.118069 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd045697-1223-445b-8123-91fc8aa77cfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:16.118376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd045697-1223-445b-8123-91fc8aa77cfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:16.119114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d8859-a7a5-4f26-8a4c-a96b94e9003a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.120059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d8859-a7a5-4f26-8a4c-a96b94e9003a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.120720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d8859-a7a5-4f26-8a4c-a96b94e9003a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.125697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd045697-1223-445b-8123-91fc8aa77cfa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:16.144196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 389/794] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:16.144196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd045697-1223-445b-8123-91fc8aa77cfa None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:16.145636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7d6d8859-a7a5-4f26-8a4c-a96b94e9003a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:16.146991 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7d6d8859-a7a5-4f26-8a4c-a96b94e9003a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:16.228129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 406/795] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:16.228129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d8859-a7a5-4f26-8a4c-a96b94e9003a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:16.263200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:16.287055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd045697-1223-445b-8123-91fc8aa77cfa None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:16.308402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:16.323873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13f544-ccbe-4fd4-8f90-3d7c6d5be214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:16.324525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13f544-ccbe-4fd4-8f90-3d7c6d5be214 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.326342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13f544-ccbe-4fd4-8f90-3d7c6d5be214 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.326342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13f544-ccbe-4fd4-8f90-3d7c6d5be214 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.326342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13f544-ccbe-4fd4-8f90-3d7c6d5be214 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:16.326342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13f544-ccbe-4fd4-8f90-3d7c6d5be214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:16.327415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13f544-ccbe-4fd4-8f90-3d7c6d5be214 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:16.342604 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 390/796] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:20:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:16.342604 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13f544-ccbe-4fd4-8f90-3d7c6d5be214 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:16.357795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d8859-a7a5-4f26-8a4c-a96b94e9003a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:16.397297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a317f-ffec-400d-be2b-52f9bef2a940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:16.398374 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a317f-ffec-400d-be2b-52f9bef2a940 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.398985 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a317f-ffec-400d-be2b-52f9bef2a940 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.399409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a317f-ffec-400d-be2b-52f9bef2a940 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.408225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a317f-ffec-400d-be2b-52f9bef2a940 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:16.409958 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a317f-ffec-400d-be2b-52f9bef2a940 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b53a317f-ffec-400d-be2b-52f9bef2a940 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:16.423383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:16.424356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:16.425287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:16.425287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:16.425287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:16.425287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:16.425287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:16.425287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:16.425287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:16.425287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:16.427891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 407/797] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:16.428880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a317f-ffec-400d-be2b-52f9bef2a940 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:16.446737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13f544-ccbe-4fd4-8f90-3d7c6d5be214 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:16.483210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02707d63-658a-4980-aa5f-dee1d75335ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:16.484041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02707d63-658a-4980-aa5f-dee1d75335ab None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.484041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02707d63-658a-4980-aa5f-dee1d75335ab None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.484304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02707d63-658a-4980-aa5f-dee1d75335ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.495494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02707d63-658a-4980-aa5f-dee1d75335ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-02707d63-658a-4980-aa5f-dee1d75335ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:16.500105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:16.501063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:16.501910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 391/798] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:16.502635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02707d63-658a-4980-aa5f-dee1d75335ab None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:16.539110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:16.546885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:16.550031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a317f-ffec-400d-be2b-52f9bef2a940 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:16.587419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd26620-e149-4aa3-9100-f4921a5a32bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:16.589121 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd26620-e149-4aa3-9100-f4921a5a32bf None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.589601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd26620-e149-4aa3-9100-f4921a5a32bf None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.590066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd26620-e149-4aa3-9100-f4921a5a32bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.602838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd26620-e149-4aa3-9100-f4921a5a32bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:16.604923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd26620-e149-4aa3-9100-f4921a5a32bf None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:16.605749 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02707d63-658a-4980-aa5f-dee1d75335ab None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-7fd26620-e149-4aa3-9100-f4921a5a32bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:16.622340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:16.623405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:16.624357 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:16.624357 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:16.624357 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:16.631231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 408/799] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:16.633431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd26620-e149-4aa3-9100-f4921a5a32bf None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:16.642505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8e9b6-b0ee-4654-a2e6-3c9b6aa1ed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:16.643187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8e9b6-b0ee-4654-a2e6-3c9b6aa1ed90 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.643529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8e9b6-b0ee-4654-a2e6-3c9b6aa1ed90 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.643974 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8e9b6-b0ee-4654-a2e6-3c9b6aa1ed90 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.644805 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8e9b6-b0ee-4654-a2e6-3c9b6aa1ed90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:16.645172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8e9b6-b0ee-4654-a2e6-3c9b6aa1ed90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:16.646658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8e9b6-b0ee-4654-a2e6-3c9b6aa1ed90 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:16.658960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 392/800] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:16.659917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8e9b6-b0ee-4654-a2e6-3c9b6aa1ed90 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:16.754844 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd26620-e149-4aa3-9100-f4921a5a32bf None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:16.764897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8e9b6-b0ee-4654-a2e6-3c9b6aa1ed90 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:16.766429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:16.790646 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31798f-1780-420e-bb90-b5f2c63bdf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:16.791966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31798f-1780-420e-bb90-b5f2c63bdf5a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.792370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31798f-1780-420e-bb90-b5f2c63bdf5a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.792646 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31798f-1780-420e-bb90-b5f2c63bdf5a None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.793313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31798f-1780-420e-bb90-b5f2c63bdf5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:16.793619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31798f-1780-420e-bb90-b5f2c63bdf5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:16.795213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31798f-1780-420e-bb90-b5f2c63bdf5a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:16.809034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 409/801] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:20:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:16.809034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31798f-1780-420e-bb90-b5f2c63bdf5a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:16.811531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ea539-35bd-4f1a-8402-f42e77ef1087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:16.812184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ea539-35bd-4f1a-8402-f42e77ef1087 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.812468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ea539-35bd-4f1a-8402-f42e77ef1087 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.812747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ea539-35bd-4f1a-8402-f42e77ef1087 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.813396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ea539-35bd-4f1a-8402-f42e77ef1087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:16.813702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ea539-35bd-4f1a-8402-f42e77ef1087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:16.815095 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ea539-35bd-4f1a-8402-f42e77ef1087 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:16.819416 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:16.829254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 393/802] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:16.838586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf0943-1260-47f1-99ac-d3030cd54918 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.838586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf0943-1260-47f1-99ac-d3030cd54918 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.838586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf0943-1260-47f1-99ac-d3030cd54918 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.838586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 394/803] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:16.838586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf0943-1260-47f1-99ac-d3030cd54918 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:16.923473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31798f-1780-420e-bb90-b5f2c63bdf5a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:16.952283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf0943-1260-47f1-99ac-d3030cd54918 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:16.967826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b425eb7-4d2c-41ab-a204-47cc501fcb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:16.968592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b425eb7-4d2c-41ab-a204-47cc501fcb45 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.968962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b425eb7-4d2c-41ab-a204-47cc501fcb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.969291 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b425eb7-4d2c-41ab-a204-47cc501fcb45 None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/e86991778c2b47c58f10d9d9e3ca5e82/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:16.971273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b425eb7-4d2c-41ab-a204-47cc501fcb45 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=e86991778c2b47c58f10d9d9e3ca5e82, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:16.974988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b425eb7-4d2c-41ab-a204-47cc501fcb45 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:16.989129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb161b-d8ee-4e53-8173-361534977158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:16.992939 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb161b-d8ee-4e53-8173-361534977158 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:16.992939 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb161b-d8ee-4e53-8173-361534977158 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:16.992939 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb161b-d8ee-4e53-8173-361534977158 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.000592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb161b-d8ee-4e53-8173-361534977158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:17.002007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb161b-d8ee-4e53-8173-361534977158 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-11fb161b-d8ee-4e53-8173-361534977158 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:17.016726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:17.017938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:17.019050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:17.019050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:17.019050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:17.020307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 395/804] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:17.021436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb161b-d8ee-4e53-8173-361534977158 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:17.029183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 410/805] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:20:16 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/e86991778c2b47c58f10d9d9e3ca5e82/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:20:17.030644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8252e8-4223-4e64-8f37-47120c09384a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:17.031082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8252e8-4223-4e64-8f37-47120c09384a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:17.031509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8252e8-4223-4e64-8f37-47120c09384a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.045341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:17.052645 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1e8252e8-4223-4e64-8f37-47120c09384a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:17.054163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1e8252e8-4223-4e64-8f37-47120c09384a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:17.132507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:17.143803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 411/806] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:17.146200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8252e8-4223-4e64-8f37-47120c09384a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:17.163927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb161b-d8ee-4e53-8173-361534977158 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:17.204379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9618ba9-d54a-492f-8584-1486a8f2d64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:17.205307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9618ba9-d54a-492f-8584-1486a8f2d64b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:17.205588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9618ba9-d54a-492f-8584-1486a8f2d64b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:17.206023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9618ba9-d54a-492f-8584-1486a8f2d64b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.217744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9618ba9-d54a-492f-8584-1486a8f2d64b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:17.219904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9618ba9-d54a-492f-8584-1486a8f2d64b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-b9618ba9-d54a-492f-8584-1486a8f2d64b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:17.235989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:17.237141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:17.238065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:17.238065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:17.238065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:17.238065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:17.238065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:17.238065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:17.238065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:17.238065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:17.238065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:17.238065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:17.238608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 396/807] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 22:20:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:17.239145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9618ba9-d54a-492f-8584-1486a8f2d64b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:17.286676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8252e8-4223-4e64-8f37-47120c09384a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:17.324882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd6df8-0607-450c-b2f5-7d15a83bccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:17.325376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:17.325512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd6df8-0607-450c-b2f5-7d15a83bccf5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:17.325750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd6df8-0607-450c-b2f5-7d15a83bccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:17.326020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd6df8-0607-450c-b2f5-7d15a83bccf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.326658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd6df8-0607-450c-b2f5-7d15a83bccf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:17.326961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd6df8-0607-450c-b2f5-7d15a83bccf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:17.328771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd6df8-0607-450c-b2f5-7d15a83bccf5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:17.341348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 412/808] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:17.342158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd6df8-0607-450c-b2f5-7d15a83bccf5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:17.355800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9618ba9-d54a-492f-8584-1486a8f2d64b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:17.391063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb770c7-d0b2-48f5-bbff-9defdd96383e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:17.391686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb770c7-d0b2-48f5-bbff-9defdd96383e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:17.391958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb770c7-d0b2-48f5-bbff-9defdd96383e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:17.392255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb770c7-d0b2-48f5-bbff-9defdd96383e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.392841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb770c7-d0b2-48f5-bbff-9defdd96383e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:17.394200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb770c7-d0b2-48f5-bbff-9defdd96383e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:17.405470 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-8cb770c7-d0b2-48f5-bbff-9defdd96383e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:17.405630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:17.406759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 22:20:17.407934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:17.411282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 397/809] 10.222.0.30 () {56 vars in 1129 bytes} [Sun Aug 7 22:20:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:17.412751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354de82-43e1-48f7-bd8a-7462f4e668f5 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:17.413041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354de82-43e1-48f7-bd8a-7462f4e668f5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:17.413318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354de82-43e1-48f7-bd8a-7462f4e668f5 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.414715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 398/810] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:17.419427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354de82-43e1-48f7-bd8a-7462f4e668f5 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:17.459013 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd6df8-0607-450c-b2f5-7d15a83bccf5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:17.501659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342c3dc-3107-41bb-9dbc-3b6b932ad454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:17.502356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342c3dc-3107-41bb-9dbc-3b6b932ad454 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:17.502692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342c3dc-3107-41bb-9dbc-3b6b932ad454 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:17.503029 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342c3dc-3107-41bb-9dbc-3b6b932ad454 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.503762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342c3dc-3107-41bb-9dbc-3b6b932ad454 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:17.504102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342c3dc-3107-41bb-9dbc-3b6b932ad454 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:17.505551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342c3dc-3107-41bb-9dbc-3b6b932ad454 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:17.528059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 413/811] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 22:20:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:17.528059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342c3dc-3107-41bb-9dbc-3b6b932ad454 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:17.533680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354de82-43e1-48f7-bd8a-7462f4e668f5 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:17.570788 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558270a-4191-4392-a526-f4e4614c4652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:17.571326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558270a-4191-4392-a526-f4e4614c4652 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:17.571602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558270a-4191-4392-a526-f4e4614c4652 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:17.571894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558270a-4191-4392-a526-f4e4614c4652 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.579847 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558270a-4191-4392-a526-f4e4614c4652 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:17.581298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558270a-4191-4392-a526-f4e4614c4652 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:17.583194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-7558270a-4191-4392-a526-f4e4614c4652 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:17.594797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:17.601424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 399/812] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 22:20:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f0e96-48ea-4d69-869a-408ce8200b60 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:17.602392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f0e96-48ea-4d69-869a-408ce8200b60 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:17.603121 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f0e96-48ea-4d69-869a-408ce8200b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.618992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6c1f0e96-48ea-4d69-869a-408ce8200b60 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:17.620525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6c1f0e96-48ea-4d69-869a-408ce8200b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:17.639531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342c3dc-3107-41bb-9dbc-3b6b932ad454 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:17.656261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:17.680525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac48e6-19fa-46b7-82ef-75f09fe03f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:17.681223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac48e6-19fa-46b7-82ef-75f09fe03f2f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:17.681635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac48e6-19fa-46b7-82ef-75f09fe03f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:17.681965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac48e6-19fa-46b7-82ef-75f09fe03f2f None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.682659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac48e6-19fa-46b7-82ef-75f09fe03f2f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:17.683029 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac48e6-19fa-46b7-82ef-75f09fe03f2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:17.684651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac48e6-19fa-46b7-82ef-75f09fe03f2f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:17.694850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 414/813] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 22:20:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:17.694850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac48e6-19fa-46b7-82ef-75f09fe03f2f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:17.699920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 400/814] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:17.700564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f0e96-48ea-4d69-869a-408ce8200b60 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:17.807577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f0e96-48ea-4d69-869a-408ce8200b60 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:17.821159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac48e6-19fa-46b7-82ef-75f09fe03f2f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:17.837543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:17.845740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19876c1e-5bc6-4c57-91b5-dd45791f02b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:17.846429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19876c1e-5bc6-4c57-91b5-dd45791f02b9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:17.853671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19876c1e-5bc6-4c57-91b5-dd45791f02b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:17.853671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19876c1e-5bc6-4c57-91b5-dd45791f02b9 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.853671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19876c1e-5bc6-4c57-91b5-dd45791f02b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:17.853671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19876c1e-5bc6-4c57-91b5-dd45791f02b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:17.853671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19876c1e-5bc6-4c57-91b5-dd45791f02b9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:17.858778 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19a09e-b43c-46d4-b1cc-18767c2ee0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:17.858919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19a09e-b43c-46d4-b1cc-18767c2ee0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:17.859377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19a09e-b43c-46d4-b1cc-18767c2ee0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:17.859713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19a09e-b43c-46d4-b1cc-18767c2ee0a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:17.872217 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 401/815] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 22:20:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:17.872217 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19876c1e-5bc6-4c57-91b5-dd45791f02b9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:17.876935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19a09e-b43c-46d4-b1cc-18767c2ee0a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:17.878456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19a09e-b43c-46d4-b1cc-18767c2ee0a1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ef19a09e-b43c-46d4-b1cc-18767c2ee0a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:17.893555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:17.894499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:17.895444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:17.895444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:17.895444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:17.895444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 415/816] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:17.895444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19a09e-b43c-46d4-b1cc-18767c2ee0a1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:17.927165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:17.984853 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19876c1e-5bc6-4c57-91b5-dd45791f02b9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:18.005559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19a09e-b43c-46d4-b1cc-18767c2ee0a1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:18.020107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184eef6-de2c-428c-83e0-bc41dd99b1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:18.020794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184eef6-de2c-428c-83e0-bc41dd99b1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:18.021135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184eef6-de2c-428c-83e0-bc41dd99b1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:18.021470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184eef6-de2c-428c-83e0-bc41dd99b1a9 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:18.022162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184eef6-de2c-428c-83e0-bc41dd99b1a9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:18.022526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184eef6-de2c-428c-83e0-bc41dd99b1a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:18.023937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184eef6-de2c-428c-83e0-bc41dd99b1a9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:18.033314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 402/817] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 22:20:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:18.033314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184eef6-de2c-428c-83e0-bc41dd99b1a9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:18.046617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92980e-526e-4cb8-983f-89f80e948c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:18.047321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92980e-526e-4cb8-983f-89f80e948c7e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:18.047670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92980e-526e-4cb8-983f-89f80e948c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:18.048009 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92980e-526e-4cb8-983f-89f80e948c7e None admin] PATH_INFO: `/v3/services/network` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:18.048693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92980e-526e-4cb8-983f-89f80e948c7e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:18.050046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92980e-526e-4cb8-983f-89f80e948c7e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-5f92980e-526e-4cb8-983f-89f80e948c7e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:18.061284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:18.062518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 22:20:18.063544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:18.067455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 416/818] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:17 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:18.068301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92980e-526e-4cb8-983f-89f80e948c7e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:18.085754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:18.139293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184eef6-de2c-428c-83e0-bc41dd99b1a9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:18.172428 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:18.178693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a4811-40f3-45e8-861d-be07ec70732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:18.179286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a4811-40f3-45e8-861d-be07ec70732d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:18.179538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92980e-526e-4cb8-983f-89f80e948c7e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:18.190720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a4811-40f3-45e8-861d-be07ec70732d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:18.190720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a4811-40f3-45e8-861d-be07ec70732d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:18.215157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d43a27-72ea-45c1-9a76-424616e403c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:18.215747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d43a27-72ea-45c1-9a76-424616e403c4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:18.216039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d43a27-72ea-45c1-9a76-424616e403c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:18.216330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d43a27-72ea-45c1-9a76-424616e403c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:18.216970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d43a27-72ea-45c1-9a76-424616e403c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:18.217281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d43a27-72ea-45c1-9a76-424616e403c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:18.218649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d43a27-72ea-45c1-9a76-424616e403c4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:18.234959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 417/819] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:18.235623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d43a27-72ea-45c1-9a76-424616e403c4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:18.252183 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a4811-40f3-45e8-861d-be07ec70732d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:18.253637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a4811-40f3-45e8-861d-be07ec70732d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:18.302256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:18.345406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d43a27-72ea-45c1-9a76-424616e403c4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:18.359993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 403/820] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:20:18.359993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a4811-40f3-45e8-861d-be07ec70732d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:18.386511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0fd77-dc52-4bf9-aa43-37ee4207f73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:18.387324 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0fd77-dc52-4bf9-aa43-37ee4207f73d None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:18.387751 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0fd77-dc52-4bf9-aa43-37ee4207f73d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:18.388210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0fd77-dc52-4bf9-aa43-37ee4207f73d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:18.388899 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0fd77-dc52-4bf9-aa43-37ee4207f73d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:18.390237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0fd77-dc52-4bf9-aa43-37ee4207f73d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:18.459146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a4811-40f3-45e8-861d-be07ec70732d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:18.496050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 418/821] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:20:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:18.496050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0fd77-dc52-4bf9-aa43-37ee4207f73d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:18.508288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d4cf3-1229-4c73-a39f-642981a41881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:18.509046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d4cf3-1229-4c73-a39f-642981a41881 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:18.509332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d4cf3-1229-4c73-a39f-642981a41881 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:18.510239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d4cf3-1229-4c73-a39f-642981a41881 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:18.511330 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d4cf3-1229-4c73-a39f-642981a41881 None admin] RBAC: Adding query filter params (name=network) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:18.511651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d4cf3-1229-4c73-a39f-642981a41881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:18.513410 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d4cf3-1229-4c73-a39f-642981a41881 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:18.515079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:18.519808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 404/822] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:18.520756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d4cf3-1229-4c73-a39f-642981a41881 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:18.536438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:18.635262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0fd77-dc52-4bf9-aa43-37ee4207f73d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:18.646404 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d4cf3-1229-4c73-a39f-642981a41881 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:18.684700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24dad6-59ea-4b5e-b3e7-d2410612c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:18.685339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24dad6-59ea-4b5e-b3e7-d2410612c311 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:18.685625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24dad6-59ea-4b5e-b3e7-d2410612c311 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:18.685906 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24dad6-59ea-4b5e-b3e7-d2410612c311 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:18.688064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e1693-394e-4a15-b34b-8520a2edd9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:18.688807 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e1693-394e-4a15-b34b-8520a2edd9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:18.689167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e1693-394e-4a15-b34b-8520a2edd9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:18.689503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e1693-394e-4a15-b34b-8520a2edd9a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:18.693870 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24dad6-59ea-4b5e-b3e7-d2410612c311 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:18.695468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24dad6-59ea-4b5e-b3e7-d2410612c311 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:18.701462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e1693-394e-4a15-b34b-8520a2edd9a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:18.703742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e1693-394e-4a15-b34b-8520a2edd9a0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-5c24dad6-59ea-4b5e-b3e7-d2410612c311 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:18.708593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:18.709691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:18.710747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:18.710747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:18.710747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:18.710747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:18.710747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:18.710747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:18.710747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:18.710747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:18.710747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 419/823] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:18.716574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24dad6-59ea-4b5e-b3e7-d2410612c311 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-893e1693-394e-4a15-b34b-8520a2edd9a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:18.720881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:18.721832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:18.722866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:18.722866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:18.722866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:18.722866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 405/824] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:18.722866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e1693-394e-4a15-b34b-8520a2edd9a0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:18.800159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:18.810949 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:18.850810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24dad6-59ea-4b5e-b3e7-d2410612c311 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:18.851881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e1693-394e-4a15-b34b-8520a2edd9a0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:18.895433 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe27c3-6bde-468b-b83c-cc147117f4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:18.896631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe27c3-6bde-468b-b83c-cc147117f4eb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:18.897124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe27c3-6bde-468b-b83c-cc147117f4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:18.897603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe27c3-6bde-468b-b83c-cc147117f4eb None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:18.898580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe27c3-6bde-468b-b83c-cc147117f4eb None admin] RBAC: Adding query filter params (type=network) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:18.899093 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe27c3-6bde-468b-b83c-cc147117f4eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:18.901052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe27c3-6bde-468b-b83c-cc147117f4eb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:18.901579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e180298-11c9-4a5c-b2ed-9db8b92be295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:18.902191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e180298-11c9-4a5c-b2ed-9db8b92be295 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:18.902470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e180298-11c9-4a5c-b2ed-9db8b92be295 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:18.902823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e180298-11c9-4a5c-b2ed-9db8b92be295 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:18.903705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e180298-11c9-4a5c-b2ed-9db8b92be295 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:18.903989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e180298-11c9-4a5c-b2ed-9db8b92be295 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:18.905504 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e180298-11c9-4a5c-b2ed-9db8b92be295 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:18.908876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 420/825] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:18.910001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe27c3-6bde-468b-b83c-cc147117f4eb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:18.924116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 406/826] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:18.924116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e180298-11c9-4a5c-b2ed-9db8b92be295 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:19.025258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e180298-11c9-4a5c-b2ed-9db8b92be295 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:19.042715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe27c3-6bde-468b-b83c-cc147117f4eb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:19.049145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:19.060750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530138d-bd21-40d0-8433-3bed6f9703a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:19.061364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530138d-bd21-40d0-8433-3bed6f9703a3 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:19.061642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530138d-bd21-40d0-8433-3bed6f9703a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:19.061917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530138d-bd21-40d0-8433-3bed6f9703a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:19.062584 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530138d-bd21-40d0-8433-3bed6f9703a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:19.071597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530138d-bd21-40d0-8433-3bed6f9703a3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:19.082449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bab15-dfc2-443a-92a0-4e11a75d8ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:19.083169 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bab15-dfc2-443a-92a0-4e11a75d8ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:19.083526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bab15-dfc2-443a-92a0-4e11a75d8ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:19.083878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bab15-dfc2-443a-92a0-4e11a75d8ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:19.084631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bab15-dfc2-443a-92a0-4e11a75d8ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:19.085008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bab15-dfc2-443a-92a0-4e11a75d8ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:19.086454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bab15-dfc2-443a-92a0-4e11a75d8ff0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:19.088654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:19.104317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 421/827] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:19.105264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bab15-dfc2-443a-92a0-4e11a75d8ff0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:19.142287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 407/828] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:20:18 2022] POST /identity/v3/endpoints => generated 337 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:19.143845 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ef129-67bc-4973-ace2-f9980ecace1f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:19.144411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ef129-67bc-4973-ace2-f9980ecace1f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:19.144896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ef129-67bc-4973-ace2-f9980ecace1f None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:19.146730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 408/829] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:19.147740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ef129-67bc-4973-ace2-f9980ecace1f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:19.226224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bab15-dfc2-443a-92a0-4e11a75d8ff0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:19.277000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ef129-67bc-4973-ace2-f9980ecace1f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:19.277208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ee272-458a-45a8-a3f6-5438a922fbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:19.277799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ee272-458a-45a8-a3f6-5438a922fbf0 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:19.278083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ee272-458a-45a8-a3f6-5438a922fbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:19.278354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ee272-458a-45a8-a3f6-5438a922fbf0 None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/77cb38c8c0dc4de3bbdeb611ffd53ff3/roles/900062fd06bc4f00add25eb39325653f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:19.304042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ee272-458a-45a8-a3f6-5438a922fbf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=77cb38c8c0dc4de3bbdeb611ffd53ff3, role_id=900062fd06bc4f00add25eb39325653f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:19.304042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ee272-458a-45a8-a3f6-5438a922fbf0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:19.324349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e275d-98ba-4073-9047-9aebe9a5b12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:19.324993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e275d-98ba-4073-9047-9aebe9a5b12e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:19.325272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e275d-98ba-4073-9047-9aebe9a5b12e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:19.325549 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e275d-98ba-4073-9047-9aebe9a5b12e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:19.333333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e275d-98ba-4073-9047-9aebe9a5b12e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:19.334779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e275d-98ba-4073-9047-9aebe9a5b12e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-240e275d-98ba-4073-9047-9aebe9a5b12e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:19.347702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:19.349592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:19.350577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:19.350577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:19.350577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:19.350577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:19.350577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:19.350577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:19.350577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:19.350577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:19.350577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 409/830] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:19.350577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b9a3c-c416-4d5c-ad15-e99bb6ac56c8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:19.350577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b9a3c-c416-4d5c-ad15-e99bb6ac56c8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:19.351184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b9a3c-c416-4d5c-ad15-e99bb6ac56c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:19.370018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-774b9a3c-c416-4d5c-ad15-e99bb6ac56c8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:19.371469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-774b9a3c-c416-4d5c-ad15-e99bb6ac56c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:19.372272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 422/831] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:20:19 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/77cb38c8c0dc4de3bbdeb611ffd53ff3/roles/900062fd06bc4f00add25eb39325653f => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:20:19.373142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ee272-458a-45a8-a3f6-5438a922fbf0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:19.404102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:19.432161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:19.468927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 410/832] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:19.517266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b9a3c-c416-4d5c-ad15-e99bb6ac56c8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:19.522854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ee272-458a-45a8-a3f6-5438a922fbf0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:19.564465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696b25f-d79d-4bca-9597-6233be47f349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:19.565325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696b25f-d79d-4bca-9597-6233be47f349 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:19.565777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696b25f-d79d-4bca-9597-6233be47f349 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:19.566192 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696b25f-d79d-4bca-9597-6233be47f349 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:19.567031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696b25f-d79d-4bca-9597-6233be47f349 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:19.567480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696b25f-d79d-4bca-9597-6233be47f349 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:19.569127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696b25f-d79d-4bca-9597-6233be47f349 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:19.582427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 423/833] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:19.589235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696b25f-d79d-4bca-9597-6233be47f349 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:19.639258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b9a3c-c416-4d5c-ad15-e99bb6ac56c8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:19.675414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6327-4165-44dd-8d36-6ce0cc682b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:19.675967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6327-4165-44dd-8d36-6ce0cc682b6d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:19.676294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6327-4165-44dd-8d36-6ce0cc682b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:19.676568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6327-4165-44dd-8d36-6ce0cc682b6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:19.687728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf6327-4165-44dd-8d36-6ce0cc682b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-46cf6327-4165-44dd-8d36-6ce0cc682b6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:19.692348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:19.693464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:19.694528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 411/834] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:19.696768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696b25f-d79d-4bca-9597-6233be47f349 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:19.701496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:19.703495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6327-4165-44dd-8d36-6ce0cc682b6d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:19.732563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:19.746996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f2892-e423-4298-a747-421e6c027628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:19.747720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2892-e423-4298-a747-421e6c027628 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:19.748000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2892-e423-4298-a747-421e6c027628 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:19.748295 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2892-e423-4298-a747-421e6c027628 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:19.749085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f2892-e423-4298-a747-421e6c027628 None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=8effbd0670ef41f59b46feb9f0997554) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:19.749397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f2892-e423-4298-a747-421e6c027628 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:19.750976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f2892-e423-4298-a747-421e6c027628 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:19.763010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 424/835] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:19 2022] GET /identity/v3/role_assignments?user.id=8effbd0670ef41f59b46feb9f0997554&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=108664305d2546949f7327b65d4cb667 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:19.834897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6327-4165-44dd-8d36-6ce0cc682b6d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:19.875086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c0ccc-30b2-4383-85bb-88c71a0ca511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:19.876156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c0ccc-30b2-4383-85bb-88c71a0ca511 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:19.876647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c0ccc-30b2-4383-85bb-88c71a0ca511 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:19.877578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c0ccc-30b2-4383-85bb-88c71a0ca511 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:19.878986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c0ccc-30b2-4383-85bb-88c71a0ca511 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:19.879753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c0ccc-30b2-4383-85bb-88c71a0ca511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:19.881477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c0ccc-30b2-4383-85bb-88c71a0ca511 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:19.893470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 412/836] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:20:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:19.900110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f2892-e423-4298-a747-421e6c027628 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:20.019986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f2892-e423-4298-a747-421e6c027628 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:20.056589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcc788-4e2d-47b2-9cde-e4f6421f86d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:20.057490 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcc788-4e2d-47b2-9cde-e4f6421f86d5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:20.057912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcc788-4e2d-47b2-9cde-e4f6421f86d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:20.058327 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcc788-4e2d-47b2-9cde-e4f6421f86d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:20.069721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fcc788-4e2d-47b2-9cde-e4f6421f86d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:20.071377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fcc788-4e2d-47b2-9cde-e4f6421f86d5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-18fcc788-4e2d-47b2-9cde-e4f6421f86d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:20.086199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:20.087334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:20.089082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:20.089082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:20.089082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:20.089857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 425/837] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:20.095941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c0ccc-30b2-4383-85bb-88c71a0ca511 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:20.198420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:20.207924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c0ccc-30b2-4383-85bb-88c71a0ca511 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:20.243337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b448-501c-4ea1-a49b-cd085e9c6067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:20.244490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b448-501c-4ea1-a49b-cd085e9c6067 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:20.244793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b448-501c-4ea1-a49b-cd085e9c6067 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:20.245046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b448-501c-4ea1-a49b-cd085e9c6067 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:20.245936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13b448-501c-4ea1-a49b-cd085e9c6067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:20.246235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13b448-501c-4ea1-a49b-cd085e9c6067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:20.247584 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13b448-501c-4ea1-a49b-cd085e9c6067 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:20.259902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 413/838] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:20.265665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcc788-4e2d-47b2-9cde-e4f6421f86d5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:20.292623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:20.375216 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcc788-4e2d-47b2-9cde-e4f6421f86d5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:20.410829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cc66b-bf08-4983-b28f-cee2dccb6429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:20.411830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cc66b-bf08-4983-b28f-cee2dccb6429 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:20.412372 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cc66b-bf08-4983-b28f-cee2dccb6429 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:20.412847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cc66b-bf08-4983-b28f-cee2dccb6429 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:20.421250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54cc66b-bf08-4983-b28f-cee2dccb6429 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:20.422965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54cc66b-bf08-4983-b28f-cee2dccb6429 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c54cc66b-bf08-4983-b28f-cee2dccb6429 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:20.435620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:20.436635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:20.437577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:20.437577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:20.437577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:20.437577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:20.437577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:20.437577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:20.437577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:20.437577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:20.437577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 22:20:20.437577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:20.444111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 426/839] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:20.445920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b448-501c-4ea1-a49b-cd085e9c6067 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:20.552423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b448-501c-4ea1-a49b-cd085e9c6067 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:20.590104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63d4a6-6e98-4634-afd4-9f1f6b4cc00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:20.590716 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63d4a6-6e98-4634-afd4-9f1f6b4cc00b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:20.590995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63d4a6-6e98-4634-afd4-9f1f6b4cc00b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:20.591322 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63d4a6-6e98-4634-afd4-9f1f6b4cc00b None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:20.592072 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63d4a6-6e98-4634-afd4-9f1f6b4cc00b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:20.592422 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63d4a6-6e98-4634-afd4-9f1f6b4cc00b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:20.593835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63d4a6-6e98-4634-afd4-9f1f6b4cc00b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:20.612886 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 414/840] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 22:20:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:20.618547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cc66b-bf08-4983-b28f-cee2dccb6429 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:20.658517 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdb512-4a30-44b8-99ed-7c96874ae341 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:20.658831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdb512-4a30-44b8-99ed-7c96874ae341 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:20.659063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdb512-4a30-44b8-99ed-7c96874ae341 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:20.660747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 415/841] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:20.668290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3169-c632-42a0-a9f4-477d46e284d3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:20.668576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3169-c632-42a0-a9f4-477d46e284d3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:20.668855 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3169-c632-42a0-a9f4-477d46e284d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:20.677437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:20.693095 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-047a3169-c632-42a0-a9f4-477d46e284d3 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:20.694947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-047a3169-c632-42a0-a9f4-477d46e284d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:20.717898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:20.740480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cc66b-bf08-4983-b28f-cee2dccb6429 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:20.778799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 416/842] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:20.783096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ec628-4b59-42ee-bb68-1946864210c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:20.783900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ec628-4b59-42ee-bb68-1946864210c1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:20.784360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ec628-4b59-42ee-bb68-1946864210c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:20.784768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ec628-4b59-42ee-bb68-1946864210c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:20.804155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3169-c632-42a0-a9f4-477d46e284d3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:20.864073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ec628-4b59-42ee-bb68-1946864210c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:20.866028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ec628-4b59-42ee-bb68-1946864210c1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:20.922162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3169-c632-42a0-a9f4-477d46e284d3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:20.959006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dd47d-6d35-4891-863b-e1b0246c9a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:20.959719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:20.960179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dd47d-6d35-4891-863b-e1b0246c9a16 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:20.960258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dd47d-6d35-4891-863b-e1b0246c9a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:20.960938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dd47d-6d35-4891-863b-e1b0246c9a16 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:20.961544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dd47d-6d35-4891-863b-e1b0246c9a16 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:20.963571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dd47d-6d35-4891-863b-e1b0246c9a16 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:20.965755 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-0c1dd47d-6d35-4891-863b-e1b0246c9a16 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:20.972962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:20.973995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:20.974833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 417/843] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:20:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:20.982943 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dd47d-6d35-4891-863b-e1b0246c9a16 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:20.999943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 427/844] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:20 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:20:21.005388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ec628-4b59-42ee-bb68-1946864210c1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:21.122071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dd47d-6d35-4891-863b-e1b0246c9a16 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:21.122596 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ec628-4b59-42ee-bb68-1946864210c1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:21.159743 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12938b-11a0-437e-b2af-7132ab30ae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:21.160408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12938b-11a0-437e-b2af-7132ab30ae68 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:21.160693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12938b-11a0-437e-b2af-7132ab30ae68 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:21.160970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12938b-11a0-437e-b2af-7132ab30ae68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:21.169691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16cff7-c921-413f-a146-c12377414541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:21.171118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16cff7-c921-413f-a146-c12377414541 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:21.171751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16cff7-c921-413f-a146-c12377414541 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:21.172329 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16cff7-c921-413f-a146-c12377414541 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:21.172838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12938b-11a0-437e-b2af-7132ab30ae68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:21.173445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16cff7-c921-413f-a146-c12377414541 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:21.173930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16cff7-c921-413f-a146-c12377414541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:21.174511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12938b-11a0-437e-b2af-7132ab30ae68 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:21.175871 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16cff7-c921-413f-a146-c12377414541 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:21.186859 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 418/845] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:20:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:21.189011 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16cff7-c921-413f-a146-c12377414541 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ee12938b-11a0-437e-b2af-7132ab30ae68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:21.194817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:21.195753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:21.196710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:21.196710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:21.196710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:21.200455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 428/846] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:21.202902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12938b-11a0-437e-b2af-7132ab30ae68 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:21.237255 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:21.275975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:21.314134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12938b-11a0-437e-b2af-7132ab30ae68 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:21.327990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16cff7-c921-413f-a146-c12377414541 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:21.352289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c89847-5bb5-4314-84ab-f12004899b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:21.352903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c89847-5bb5-4314-84ab-f12004899b9b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:21.353237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c89847-5bb5-4314-84ab-f12004899b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:21.353609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c89847-5bb5-4314-84ab-f12004899b9b None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:21.354523 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c89847-5bb5-4314-84ab-f12004899b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:21.354929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c89847-5bb5-4314-84ab-f12004899b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:21.356508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c89847-5bb5-4314-84ab-f12004899b9b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:21.373024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447921a-8148-46fb-9af5-38d794b43ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:21.374087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 429/847] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:21.374385 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447921a-8148-46fb-9af5-38d794b43ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:21.374864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447921a-8148-46fb-9af5-38d794b43ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:21.374981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c89847-5bb5-4314-84ab-f12004899b9b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:21.375450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447921a-8148-46fb-9af5-38d794b43ce5 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:21.376318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447921a-8148-46fb-9af5-38d794b43ce5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:21.376822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447921a-8148-46fb-9af5-38d794b43ce5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:21.379600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447921a-8148-46fb-9af5-38d794b43ce5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:21.392038 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 419/848] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:20:21 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:21.487257 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:21.494797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7358f-a282-416e-b33d-f9a6f8830ebb None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:21.495171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7358f-a282-416e-b33d-f9a6f8830ebb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:21.495835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7358f-a282-416e-b33d-f9a6f8830ebb None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:21.497600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 420/849] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:21.502337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c89847-5bb5-4314-84ab-f12004899b9b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:21.505063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d732cf-b367-4848-98e0-b7e8211551f6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:21.505349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d732cf-b367-4848-98e0-b7e8211551f6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:21.505630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d732cf-b367-4848-98e0-b7e8211551f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:21.530515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-74d732cf-b367-4848-98e0-b7e8211551f6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:21.532301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-74d732cf-b367-4848-98e0-b7e8211551f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:21.543814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a4ebc-5a95-4f9f-8177-04b1c65c7f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:21.544905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a4ebc-5a95-4f9f-8177-04b1c65c7f24 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:21.545290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a4ebc-5a95-4f9f-8177-04b1c65c7f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:21.545668 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a4ebc-5a95-4f9f-8177-04b1c65c7f24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:21.553640 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a4ebc-5a95-4f9f-8177-04b1c65c7f24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:21.555220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a4ebc-5a95-4f9f-8177-04b1c65c7f24 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-6e9a4ebc-5a95-4f9f-8177-04b1c65c7f24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:21.567995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:21.569028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:21.569915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:21.569915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:21.569915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:21.569915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:21.569915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:21.569915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:21.569915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:21.569915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:21.574042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 430/850] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:21.575950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a4ebc-5a95-4f9f-8177-04b1c65c7f24 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:21.638215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 421/851] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:21.657321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d732cf-b367-4848-98e0-b7e8211551f6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:21.702565 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a4ebc-5a95-4f9f-8177-04b1c65c7f24 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:21.724420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:21.747085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d504a-6a78-4968-a61b-134e7461aefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:21.748088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d504a-6a78-4968-a61b-134e7461aefe None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:21.748334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d504a-6a78-4968-a61b-134e7461aefe None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:21.748897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d504a-6a78-4968-a61b-134e7461aefe None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:21.749670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d504a-6a78-4968-a61b-134e7461aefe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:21.749984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d504a-6a78-4968-a61b-134e7461aefe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:21.752111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d504a-6a78-4968-a61b-134e7461aefe None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:21.768754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 431/852] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:21.774590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d504a-6a78-4968-a61b-134e7461aefe None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:21.777607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d732cf-b367-4848-98e0-b7e8211551f6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:21.792745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:21.824495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b365e2-f968-49c6-b12a-5796d68b030c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:21.825283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b365e2-f968-49c6-b12a-5796d68b030c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:21.825664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b365e2-f968-49c6-b12a-5796d68b030c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:21.826040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b365e2-f968-49c6-b12a-5796d68b030c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:21.838130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b365e2-f968-49c6-b12a-5796d68b030c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-37b365e2-f968-49c6-b12a-5796d68b030c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:21.843297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:21.844452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:20:21.845502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:21.851913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 422/853] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:20:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:21.853295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b365e2-f968-49c6-b12a-5796d68b030c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:21.902180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d504a-6a78-4968-a61b-134e7461aefe None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:21.950720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d050bb7-8fbc-4b81-a2af-75397499360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:21.951713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d050bb7-8fbc-4b81-a2af-75397499360c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:21.952086 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d050bb7-8fbc-4b81-a2af-75397499360c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:21.952701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d050bb7-8fbc-4b81-a2af-75397499360c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:21.953893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d050bb7-8fbc-4b81-a2af-75397499360c None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=e86991778c2b47c58f10d9d9e3ca5e82) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:21.954280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d050bb7-8fbc-4b81-a2af-75397499360c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:21.955795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d050bb7-8fbc-4b81-a2af-75397499360c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:21.968197 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 432/854] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:21 2022] GET /identity/v3/role_assignments?user.id=e86991778c2b47c58f10d9d9e3ca5e82&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=108664305d2546949f7327b65d4cb667 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:21.984610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b365e2-f968-49c6-b12a-5796d68b030c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:22.005812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:22.025808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deacd7-542b-4a62-8b76-02208c37a842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:22.031309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deacd7-542b-4a62-8b76-02208c37a842 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:22.031309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deacd7-542b-4a62-8b76-02208c37a842 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:22.031309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deacd7-542b-4a62-8b76-02208c37a842 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:22.031309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1deacd7-542b-4a62-8b76-02208c37a842 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:22.031309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1deacd7-542b-4a62-8b76-02208c37a842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:22.031309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1deacd7-542b-4a62-8b76-02208c37a842 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:22.040337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 423/855] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:20:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:22.042323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fac69-c8ed-4934-9d9d-4740d6b5f9b3 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:22.042705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fac69-c8ed-4934-9d9d-4740d6b5f9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:22.043080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fac69-c8ed-4934-9d9d-4740d6b5f9b3 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:22.048555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 433/856] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:22.049978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deacd7-542b-4a62-8b76-02208c37a842 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:22.050822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a752377-8bcc-43f2-a739-8130b814054b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:22.051325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a752377-8bcc-43f2-a739-8130b814054b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:22.051704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a752377-8bcc-43f2-a739-8130b814054b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:22.074820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8a752377-8bcc-43f2-a739-8130b814054b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:22.076349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8a752377-8bcc-43f2-a739-8130b814054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:22.160390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 434/857] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:22.181755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a752377-8bcc-43f2-a739-8130b814054b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:22.185349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deacd7-542b-4a62-8b76-02208c37a842 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:22.190492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:22.235496 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f5c3c-f9e2-4c50-9596-dd889d49f06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:22.236403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f5c3c-f9e2-4c50-9596-dd889d49f06f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:22.236803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f5c3c-f9e2-4c50-9596-dd889d49f06f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:22.237179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f5c3c-f9e2-4c50-9596-dd889d49f06f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:22.249180 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f5c3c-f9e2-4c50-9596-dd889d49f06f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:22.250834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f5c3c-f9e2-4c50-9596-dd889d49f06f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ca9f5c3c-f9e2-4c50-9596-dd889d49f06f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:22.265917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:22.267047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:22.268156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:22.268156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:22.268156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:22.273851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 424/858] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:22.275264 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f5c3c-f9e2-4c50-9596-dd889d49f06f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:22.310253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:22.310891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a752377-8bcc-43f2-a739-8130b814054b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:22.356240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177edea7-df9a-4c60-8893-fb04dd37ad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:22.357097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177edea7-df9a-4c60-8893-fb04dd37ad17 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:22.357443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177edea7-df9a-4c60-8893-fb04dd37ad17 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:22.357816 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177edea7-df9a-4c60-8893-fb04dd37ad17 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:22.358505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177edea7-df9a-4c60-8893-fb04dd37ad17 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:22.360578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177edea7-df9a-4c60-8893-fb04dd37ad17 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-177edea7-df9a-4c60-8893-fb04dd37ad17 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:22.369976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:22.371775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 22:20:22.372895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:22.379334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 435/859] 10.222.0.30 () {56 vars in 1132 bytes} [Sun Aug 7 22:20:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:22.380595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177edea7-df9a-4c60-8893-fb04dd37ad17 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:22.402134 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f5c3c-f9e2-4c50-9596-dd889d49f06f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:22.449753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3b3de-d4f4-43b4-a520-49b46188c279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:22.450837 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3b3de-d4f4-43b4-a520-49b46188c279 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:22.451309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3b3de-d4f4-43b4-a520-49b46188c279 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:22.452287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3b3de-d4f4-43b4-a520-49b46188c279 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:22.453353 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3b3de-d4f4-43b4-a520-49b46188c279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:22.453831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3b3de-d4f4-43b4-a520-49b46188c279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:22.454996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:22.457502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3b3de-d4f4-43b4-a520-49b46188c279 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:22.472766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 425/860] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:22.478481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3b3de-d4f4-43b4-a520-49b46188c279 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:22.496074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177edea7-df9a-4c60-8893-fb04dd37ad17 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:22.543041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3d924-6ded-4aa1-8843-9e25fcd7d710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:22.543648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3d924-6ded-4aa1-8843-9e25fcd7d710 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:22.543926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3d924-6ded-4aa1-8843-9e25fcd7d710 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:22.544230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3d924-6ded-4aa1-8843-9e25fcd7d710 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:22.544858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3d924-6ded-4aa1-8843-9e25fcd7d710 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:22.545161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3d924-6ded-4aa1-8843-9e25fcd7d710 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:22.546526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3d924-6ded-4aa1-8843-9e25fcd7d710 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:22.551577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 436/861] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 22:20:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:22.557736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3d924-6ded-4aa1-8843-9e25fcd7d710 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:22.582637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:22.593940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3b3de-d4f4-43b4-a520-49b46188c279 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:22.632729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccf58f-7873-4cee-a6b6-51aac0b6b477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:22.633491 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccf58f-7873-4cee-a6b6-51aac0b6b477 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:22.633830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccf58f-7873-4cee-a6b6-51aac0b6b477 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:22.634165 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccf58f-7873-4cee-a6b6-51aac0b6b477 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:22.642619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccf58f-7873-4cee-a6b6-51aac0b6b477 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:22.644218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccf58f-7873-4cee-a6b6-51aac0b6b477 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-4dccf58f-7873-4cee-a6b6-51aac0b6b477 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:22.657103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:22.658188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:22.660058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:22.660058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:22.660058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:22.660058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:22.660058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:22.660058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:22.660058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:22.660058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:22.660058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:22.660058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:22.663308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 426/862] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 22:20:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:22.664809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccf58f-7873-4cee-a6b6-51aac0b6b477 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:22.675006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3d924-6ded-4aa1-8843-9e25fcd7d710 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:22.714143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78cfb6-673d-4cb4-8603-cfabd7552b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:22.714676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78cfb6-673d-4cb4-8603-cfabd7552b5d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:22.714951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78cfb6-673d-4cb4-8603-cfabd7552b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:22.715221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78cfb6-673d-4cb4-8603-cfabd7552b5d None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:22.715853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78cfb6-673d-4cb4-8603-cfabd7552b5d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:22.716289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78cfb6-673d-4cb4-8603-cfabd7552b5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:22.717748 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78cfb6-673d-4cb4-8603-cfabd7552b5d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:22.719010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:22.723702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 437/863] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 22:20:22 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:22.782808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccf58f-7873-4cee-a6b6-51aac0b6b477 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:22.829036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610a444-6330-46f3-9525-5b426dc5525d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:22.830063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610a444-6330-46f3-9525-5b426dc5525d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:22.830528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610a444-6330-46f3-9525-5b426dc5525d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:22.830974 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610a444-6330-46f3-9525-5b426dc5525d None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:22.831860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610a444-6330-46f3-9525-5b426dc5525d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:22.832353 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610a444-6330-46f3-9525-5b426dc5525d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:22.834031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610a444-6330-46f3-9525-5b426dc5525d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:22.855510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 427/864] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 22:20:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:22.865194 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78cfb6-673d-4cb4-8603-cfabd7552b5d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:22.988418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78cfb6-673d-4cb4-8603-cfabd7552b5d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:23.025986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6432cb1-5c79-4357-b6ff-109af2ab6f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:23.027000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6432cb1-5c79-4357-b6ff-109af2ab6f2d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:23.027627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6432cb1-5c79-4357-b6ff-109af2ab6f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:23.028190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6432cb1-5c79-4357-b6ff-109af2ab6f2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:23.108392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:23.112376 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6432cb1-5c79-4357-b6ff-109af2ab6f2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:23.113966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6432cb1-5c79-4357-b6ff-109af2ab6f2d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:23.265290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 438/865] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:22 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:20:23.273128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610a444-6330-46f3-9525-5b426dc5525d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:23.301915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:23.490474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610a444-6330-46f3-9525-5b426dc5525d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:23.541940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51c847-5549-4ccd-82cb-f7259b09fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:23.542683 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51c847-5549-4ccd-82cb-f7259b09fdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:23.542964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51c847-5549-4ccd-82cb-f7259b09fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:23.543238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51c847-5549-4ccd-82cb-f7259b09fdb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:23.571012 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51c847-5549-4ccd-82cb-f7259b09fdb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:23.571012 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51c847-5549-4ccd-82cb-f7259b09fdb9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-8f51c847-5549-4ccd-82cb-f7259b09fdb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:23.590589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:23.591637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:23.592565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:23.592565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:23.592565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:23.598057 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 428/866] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:23.602562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6432cb1-5c79-4357-b6ff-109af2ab6f2d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:23.661560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfdd2a-605b-4af9-9f49-4c0aa45bb666 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:23.661917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfdd2a-605b-4af9-9f49-4c0aa45bb666 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:23.662259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfdd2a-605b-4af9-9f49-4c0aa45bb666 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:23.664687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 429/867] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:23.672507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e47e1-f8fd-4fc7-8b2b-de37da1745a9 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:23.672965 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e47e1-f8fd-4fc7-8b2b-de37da1745a9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:23.673637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e47e1-f8fd-4fc7-8b2b-de37da1745a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:23.697461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-795e47e1-f8fd-4fc7-8b2b-de37da1745a9 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:23.699716 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-795e47e1-f8fd-4fc7-8b2b-de37da1745a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:23.761200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:23.766127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6432cb1-5c79-4357-b6ff-109af2ab6f2d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:23.776661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:23.800110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 430/868] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:23.815883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a3574-00e1-4f14-9ddc-d52ac9ada293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:23.816530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a3574-00e1-4f14-9ddc-d52ac9ada293 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:23.816812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a3574-00e1-4f14-9ddc-d52ac9ada293 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:23.817088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a3574-00e1-4f14-9ddc-d52ac9ada293 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:23.817736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a3574-00e1-4f14-9ddc-d52ac9ada293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:23.818044 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a3574-00e1-4f14-9ddc-d52ac9ada293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:23.819422 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a3574-00e1-4f14-9ddc-d52ac9ada293 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:23.825029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e47e1-f8fd-4fc7-8b2b-de37da1745a9 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:23.840888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 439/869] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:23.843783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a3574-00e1-4f14-9ddc-d52ac9ada293 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:23.930038 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e47e1-f8fd-4fc7-8b2b-de37da1745a9 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:23.956521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a3574-00e1-4f14-9ddc-d52ac9ada293 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:23.969600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f705976-f7b3-414a-b3c3-08118d955fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:23.970228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f705976-f7b3-414a-b3c3-08118d955fa5 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:23.970511 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f705976-f7b3-414a-b3c3-08118d955fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:23.970787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f705976-f7b3-414a-b3c3-08118d955fa5 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:23.971469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f705976-f7b3-414a-b3c3-08118d955fa5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:23.972996 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f705976-f7b3-414a-b3c3-08118d955fa5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:24.001225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb9fdb-e688-4a5e-b451-1c6308f29e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:24.002639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb9fdb-e688-4a5e-b451-1c6308f29e74 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:24.003178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb9fdb-e688-4a5e-b451-1c6308f29e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:24.003649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb9fdb-e688-4a5e-b451-1c6308f29e74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:24.011957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb9fdb-e688-4a5e-b451-1c6308f29e74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:24.018750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb9fdb-e688-4a5e-b451-1c6308f29e74 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-eefb9fdb-e688-4a5e-b451-1c6308f29e74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:24.026037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:24.026927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:24.027845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:24.027845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:24.027845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:24.027845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:24.027845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:24.027845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:24.027845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:24.027845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:24.028940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 440/870] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:24.033282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb9fdb-e688-4a5e-b451-1c6308f29e74 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:24.056100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 431/871] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:20:23 2022] POST /identity/v3/services => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:24.076423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:24.153377 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4547-8970-440f-b7d2-fc13fc8ba813 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:24.153743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4547-8970-440f-b7d2-fc13fc8ba813 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:24.153927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4547-8970-440f-b7d2-fc13fc8ba813 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:24.155542 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 432/872] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:20:24.161138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4622a0-b254-480a-964f-c7adfcac2cb0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:24.161422 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4622a0-b254-480a-964f-c7adfcac2cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:24.161701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4622a0-b254-480a-964f-c7adfcac2cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:24.166592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb9fdb-e688-4a5e-b451-1c6308f29e74 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:24.181602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9f4622a0-b254-480a-964f-c7adfcac2cb0 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:24.183052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9f4622a0-b254-480a-964f-c7adfcac2cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:24.223712 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456aa2e-333e-48c7-babb-f24f75c6d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:24.224389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456aa2e-333e-48c7-babb-f24f75c6d116 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:24.224675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456aa2e-333e-48c7-babb-f24f75c6d116 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:24.224941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456aa2e-333e-48c7-babb-f24f75c6d116 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:24.225731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456aa2e-333e-48c7-babb-f24f75c6d116 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:24.226839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456aa2e-333e-48c7-babb-f24f75c6d116 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:24.227690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456aa2e-333e-48c7-babb-f24f75c6d116 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:24.240794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 441/873] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:24.248805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456aa2e-333e-48c7-babb-f24f75c6d116 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:24.249532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:24.271472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 433/874] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:24 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:24.294288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4622a0-b254-480a-964f-c7adfcac2cb0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:24.340292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:24.368469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456aa2e-333e-48c7-babb-f24f75c6d116 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:24.413879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5890030-5797-4602-a4c1-782eb63fa9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:24.414621 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5890030-5797-4602-a4c1-782eb63fa9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:24.414906 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5890030-5797-4602-a4c1-782eb63fa9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:24.415183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5890030-5797-4602-a4c1-782eb63fa9c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:24.416583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5890030-5797-4602-a4c1-782eb63fa9c5 None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=77cb38c8c0dc4de3bbdeb611ffd53ff3) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:24.417228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5890030-5797-4602-a4c1-782eb63fa9c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:24.418844 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5890030-5797-4602-a4c1-782eb63fa9c5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:24.423638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4622a0-b254-480a-964f-c7adfcac2cb0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:24.425657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 442/875] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:24 2022] GET /identity/v3/role_assignments?user.id=77cb38c8c0dc4de3bbdeb611ffd53ff3&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:24.469939 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6f138-77af-46d3-9687-e7ed99873ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:24.470647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6f138-77af-46d3-9687-e7ed99873ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:24.470920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6f138-77af-46d3-9687-e7ed99873ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:24.471240 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6f138-77af-46d3-9687-e7ed99873ec8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:24.471848 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6f138-77af-46d3-9687-e7ed99873ec8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:24.473283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6f138-77af-46d3-9687-e7ed99873ec8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-0dc6f138-77af-46d3-9687-e7ed99873ec8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:24.483114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:24.484193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:24.485023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 434/876] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 22:20:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:24.493249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5890030-5797-4602-a4c1-782eb63fa9c5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:24.612484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5890030-5797-4602-a4c1-782eb63fa9c5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:24.649061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79595d-f8d9-4da1-9065-184eabb242bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:24.649672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79595d-f8d9-4da1-9065-184eabb242bf None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:24.650534 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79595d-f8d9-4da1-9065-184eabb242bf None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:24.650726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79595d-f8d9-4da1-9065-184eabb242bf None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:24.651360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79595d-f8d9-4da1-9065-184eabb242bf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:24.651656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79595d-f8d9-4da1-9065-184eabb242bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:24.653046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79595d-f8d9-4da1-9065-184eabb242bf None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:24.658921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 443/877] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 22:20:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:24.664844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6f138-77af-46d3-9687-e7ed99873ec8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:24.717106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:24.776482 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6f138-77af-46d3-9687-e7ed99873ec8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:24.812374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189f5cf-4207-4e79-863f-0315d9b643f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:24.812988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189f5cf-4207-4e79-863f-0315d9b643f1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:24.813267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189f5cf-4207-4e79-863f-0315d9b643f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:24.813541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189f5cf-4207-4e79-863f-0315d9b643f1 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:24.814181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189f5cf-4207-4e79-863f-0315d9b643f1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:24.814484 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189f5cf-4207-4e79-863f-0315d9b643f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:24.815852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189f5cf-4207-4e79-863f-0315d9b643f1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:24.820947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 435/878] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 22:20:24 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:24.898707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2feaf6-81b9-40ac-b4a7-10388b536130 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:24.898953 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2feaf6-81b9-40ac-b4a7-10388b536130 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:24.899186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2feaf6-81b9-40ac-b4a7-10388b536130 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:24.900916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 444/879] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:24.906628 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341545e-d1a0-4d28-b6a9-0a66b29b5678 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:24.906932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341545e-d1a0-4d28-b6a9-0a66b29b5678 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:24.907193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341545e-d1a0-4d28-b6a9-0a66b29b5678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:24.926914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e341545e-d1a0-4d28-b6a9-0a66b29b5678 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:24.928183 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e341545e-d1a0-4d28-b6a9-0a66b29b5678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:25.004076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 436/880] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:24 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:25.026912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2feaf6-81b9-40ac-b4a7-10388b536130 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:25.029834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:25.134836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2feaf6-81b9-40ac-b4a7-10388b536130 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:25.170273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b39859-ea39-41ef-bac9-9762596664af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:25.170799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b39859-ea39-41ef-bac9-9762596664af None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:25.171076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b39859-ea39-41ef-bac9-9762596664af None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:25.171346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b39859-ea39-41ef-bac9-9762596664af None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:25.172004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b39859-ea39-41ef-bac9-9762596664af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:25.173391 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b39859-ea39-41ef-bac9-9762596664af None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:25.275593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 445/881] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:20:25 2022] POST /identity/v3/services => generated 269 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:26.218945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caaa978-c640-4a36-9975-d53ebc1a3d09 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:26.219452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caaa978-c640-4a36-9975-d53ebc1a3d09 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:26.219452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caaa978-c640-4a36-9975-d53ebc1a3d09 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:26.221069 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 437/882] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:26.226259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dcb43-1e26-41d4-8581-16f6c4c810e2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:26.226646 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dcb43-1e26-41d4-8581-16f6c4c810e2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:26.226809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dcb43-1e26-41d4-8581-16f6c4c810e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:26.246112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-833dcb43-1e26-41d4-8581-16f6c4c810e2 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:26.247307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-833dcb43-1e26-41d4-8581-16f6c4c810e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:26.309146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:26.325201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 446/883] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:26.348380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caaa978-c640-4a36-9975-d53ebc1a3d09 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:26.398030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:26.465302 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caaa978-c640-4a36-9975-d53ebc1a3d09 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:26.502675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79f534-2a00-4c07-bdff-e81049bdbd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:26.503227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f534-2a00-4c07-bdff-e81049bdbd3b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:26.503499 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f534-2a00-4c07-bdff-e81049bdbd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:26.503767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f534-2a00-4c07-bdff-e81049bdbd3b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:26.504379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79f534-2a00-4c07-bdff-e81049bdbd3b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:26.505734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79f534-2a00-4c07-bdff-e81049bdbd3b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ca79f534-2a00-4c07-bdff-e81049bdbd3b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:26.514765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:26.515750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:26.516641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 438/884] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:20:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:26.527731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dcb43-1e26-41d4-8581-16f6c4c810e2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:26.631689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dcb43-1e26-41d4-8581-16f6c4c810e2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:26.669574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1a79b-6b08-480c-bfb7-33332191c82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:26.670582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1a79b-6b08-480c-bfb7-33332191c82d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:26.671069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1a79b-6b08-480c-bfb7-33332191c82d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:26.671653 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1a79b-6b08-480c-bfb7-33332191c82d None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:26.672523 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1a79b-6b08-480c-bfb7-33332191c82d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:26.673037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1a79b-6b08-480c-bfb7-33332191c82d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:26.674628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1a79b-6b08-480c-bfb7-33332191c82d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:26.733023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 447/885] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:20:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:26.738590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79f534-2a00-4c07-bdff-e81049bdbd3b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:26.762565 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb71534-8771-4f24-9c16-a6df930768de None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:26.763174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb71534-8771-4f24-9c16-a6df930768de None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:26.763527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb71534-8771-4f24-9c16-a6df930768de None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:26.765211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 448/886] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:20:26.770003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16f2de-68f9-41cd-930a-51a962712568 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:26.770283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16f2de-68f9-41cd-930a-51a962712568 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:26.770555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16f2de-68f9-41cd-930a-51a962712568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:26.789262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0d16f2de-68f9-41cd-930a-51a962712568 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:26.790449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0d16f2de-68f9-41cd-930a-51a962712568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:26.854221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:26.855896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79f534-2a00-4c07-bdff-e81049bdbd3b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:26.857659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:26.880465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 449/887] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:26.880465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb7fa7-8dd4-45e6-8762-4b63b8793c01 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:26.880465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb7fa7-8dd4-45e6-8762-4b63b8793c01 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:26.880465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb7fa7-8dd4-45e6-8762-4b63b8793c01 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:26.885550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 450/888] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:26.886885 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb67ed-01f3-46c8-8054-93f7c46ce26d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:26.887176 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb67ed-01f3-46c8-8054-93f7c46ce26d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:26.887459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb67ed-01f3-46c8-8054-93f7c46ce26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:26.895543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b5ad1-e966-4c0b-b065-6a770d65de9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:26.896375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b5ad1-e966-4c0b-b065-6a770d65de9b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:26.896792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b5ad1-e966-4c0b-b065-6a770d65de9b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:26.897204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b5ad1-e966-4c0b-b065-6a770d65de9b None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:26.898163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b5ad1-e966-4c0b-b065-6a770d65de9b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:26.898611 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b5ad1-e966-4c0b-b065-6a770d65de9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:26.900781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b5ad1-e966-4c0b-b065-6a770d65de9b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:26.906391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 439/889] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:20:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:26.907452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b5ad1-e966-4c0b-b065-6a770d65de9b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:26.907960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-55fb67ed-01f3-46c8-8054-93f7c46ce26d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:26.909271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-55fb67ed-01f3-46c8-8054-93f7c46ce26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:26.983915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 451/890] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:26.989835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb67ed-01f3-46c8-8054-93f7c46ce26d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:27.022490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b5ad1-e966-4c0b-b065-6a770d65de9b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:27.061479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44515274-e30a-4bde-856f-de358c87d332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:27.062405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515274-e30a-4bde-856f-de358c87d332 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:27.062868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515274-e30a-4bde-856f-de358c87d332 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:27.063325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515274-e30a-4bde-856f-de358c87d332 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:27.074649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44515274-e30a-4bde-856f-de358c87d332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-44515274-e30a-4bde-856f-de358c87d332 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:27.079540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:27.080476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:27.081328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:27.083260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 440/891] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 22:20:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:27.084205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44515274-e30a-4bde-856f-de358c87d332 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:27.108744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb67ed-01f3-46c8-8054-93f7c46ce26d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:27.115001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:27.125435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:27.147460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e888d-8e50-453d-9b89-1067246651d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:27.148103 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e888d-8e50-453d-9b89-1067246651d0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:27.148384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e888d-8e50-453d-9b89-1067246651d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:27.148875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e888d-8e50-453d-9b89-1067246651d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:27.149349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e888d-8e50-453d-9b89-1067246651d0 None admin] RBAC: Adding query filter params (interface=public, service_id=88759c5fce7d40b4afa8bbd28174ba33, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:27.149657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e888d-8e50-453d-9b89-1067246651d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:27.151142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e888d-8e50-453d-9b89-1067246651d0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:27.156096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 452/892] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 22:20:26 2022] GET /identity/v3/endpoints?service_id=88759c5fce7d40b4afa8bbd28174ba33&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:27.160619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e888d-8e50-453d-9b89-1067246651d0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:27.203290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44515274-e30a-4bde-856f-de358c87d332 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:27.241864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110a885-facb-416f-8f19-2fc7f73f47ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:27.242507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110a885-facb-416f-8f19-2fc7f73f47ef None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:27.243135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110a885-facb-416f-8f19-2fc7f73f47ef None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:27.243414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110a885-facb-416f-8f19-2fc7f73f47ef None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:27.244331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110a885-facb-416f-8f19-2fc7f73f47ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:27.245945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110a885-facb-416f-8f19-2fc7f73f47ef None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:27.261437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e888d-8e50-453d-9b89-1067246651d0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:27.294616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff847cd-bf2c-4c79-8aab-e07f38bfff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:27.295189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff847cd-bf2c-4c79-8aab-e07f38bfff8b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:27.295465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff847cd-bf2c-4c79-8aab-e07f38bfff8b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:27.295733 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff847cd-bf2c-4c79-8aab-e07f38bfff8b None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:27.296415 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff847cd-bf2c-4c79-8aab-e07f38bfff8b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:27.296716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff847cd-bf2c-4c79-8aab-e07f38bfff8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:27.298153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff847cd-bf2c-4c79-8aab-e07f38bfff8b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:27.308106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 441/893] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:20:27 2022] POST /identity/v3/services => generated 241 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:27.309187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110a885-facb-416f-8f19-2fc7f73f47ef None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:27.311154 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 453/894] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 22:20:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:27.312117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff847cd-bf2c-4c79-8aab-e07f38bfff8b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:27.364572 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:27.414411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:27.436852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110a885-facb-416f-8f19-2fc7f73f47ef None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:27.438247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff847cd-bf2c-4c79-8aab-e07f38bfff8b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:27.477714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280f8c7-31c3-4b9e-a906-8c7167fc43b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:27.478918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280f8c7-31c3-4b9e-a906-8c7167fc43b8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:27.479204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280f8c7-31c3-4b9e-a906-8c7167fc43b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:27.479451 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280f8c7-31c3-4b9e-a906-8c7167fc43b8 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:27.480089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280f8c7-31c3-4b9e-a906-8c7167fc43b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:27.481441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280f8c7-31c3-4b9e-a906-8c7167fc43b8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:27.482059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec66ef5-21d4-4213-a1de-df698f622b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:27.482700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec66ef5-21d4-4213-a1de-df698f622b0a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:27.483220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec66ef5-21d4-4213-a1de-df698f622b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:27.483220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec66ef5-21d4-4213-a1de-df698f622b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:27.487572 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 442/895] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 22:20:27 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:20:27.488622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804a25a-ddb4-47f8-8c52-2774c4735bc5 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:27.488904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804a25a-ddb4-47f8-8c52-2774c4735bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:27.489235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804a25a-ddb4-47f8-8c52-2774c4735bc5 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:27.490361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 443/896] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:27.494890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec66ef5-21d4-4213-a1de-df698f622b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:27.497564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7c299-b8b6-4277-9b20-6ce2d1c6e5e3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:27.497564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7c299-b8b6-4277-9b20-6ce2d1c6e5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:27.497793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec66ef5-21d4-4213-a1de-df698f622b0a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:27.497869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7c299-b8b6-4277-9b20-6ce2d1c6e5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-3ec66ef5-21d4-4213-a1de-df698f622b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:27.514540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:27.515548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:27.516488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:27.516488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:27.516488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:27.516488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 454/897] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:27.517236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6fe7c299-b8b6-4277-9b20-6ce2d1c6e5e3 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:27.518350 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6fe7c299-b8b6-4277-9b20-6ce2d1c6e5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:27.522310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec66ef5-21d4-4213-a1de-df698f622b0a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:27.602923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 444/898] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:27.625858 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7c299-b8b6-4277-9b20-6ce2d1c6e5e3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:27.660121 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:27.673543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec66ef5-21d4-4213-a1de-df698f622b0a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:27.717452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977f0c5-b4dd-4958-b338-4f65860fcf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:27.718070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f0c5-b4dd-4958-b338-4f65860fcf68 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:27.718355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f0c5-b4dd-4958-b338-4f65860fcf68 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:27.718634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f0c5-b4dd-4958-b338-4f65860fcf68 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:27.719408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977f0c5-b4dd-4958-b338-4f65860fcf68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:27.719721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977f0c5-b4dd-4958-b338-4f65860fcf68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:27.721420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:27.722206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977f0c5-b4dd-4958-b338-4f65860fcf68 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:27.734549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 455/899] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:27.741277 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977f0c5-b4dd-4958-b338-4f65860fcf68 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:27.753128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7c299-b8b6-4277-9b20-6ce2d1c6e5e3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:27.802263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b12e74-2bf6-44b2-b2fe-939a837e27b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:27.803146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b12e74-2bf6-44b2-b2fe-939a837e27b3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:27.803416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b12e74-2bf6-44b2-b2fe-939a837e27b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:27.804230 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b12e74-2bf6-44b2-b2fe-939a837e27b3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:27.804827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b12e74-2bf6-44b2-b2fe-939a837e27b3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:27.806733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b12e74-2bf6-44b2-b2fe-939a837e27b3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-74b12e74-2bf6-44b2-b2fe-939a837e27b3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:27.819594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:27.820569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:27.821456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 445/900] 10.222.0.30 () {56 vars in 1132 bytes} [Sun Aug 7 22:20:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:27.830018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b12e74-2bf6-44b2-b2fe-939a837e27b3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:27.866739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977f0c5-b4dd-4958-b338-4f65860fcf68 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:27.905836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e8864-cb11-45ef-9a44-b3a7f72d0d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:27.906471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e8864-cb11-45ef-9a44-b3a7f72d0d8c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:27.906746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e8864-cb11-45ef-9a44-b3a7f72d0d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:27.907020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e8864-cb11-45ef-9a44-b3a7f72d0d8c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:27.915256 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e8864-cb11-45ef-9a44-b3a7f72d0d8c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:27.917614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e8864-cb11-45ef-9a44-b3a7f72d0d8c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-bf3e8864-cb11-45ef-9a44-b3a7f72d0d8c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:27.930034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:27.931513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:27.933112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:27.933112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:27.933112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:27.933112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:27.933112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:27.933112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:27.933112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:27.933112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:27.933112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:27.933112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:27.933112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 456/901] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 22:20:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:27.941043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e8864-cb11-45ef-9a44-b3a7f72d0d8c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:27.957665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:27.965899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b12e74-2bf6-44b2-b2fe-939a837e27b3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:28.008793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b4e4b-7a89-4aa5-891f-c1c03202ca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:28.010060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4e4b-7a89-4aa5-891f-c1c03202ca09 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:28.010468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4e4b-7a89-4aa5-891f-c1c03202ca09 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:28.010754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4e4b-7a89-4aa5-891f-c1c03202ca09 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:28.011869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b4e4b-7a89-4aa5-891f-c1c03202ca09 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:28.012207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b4e4b-7a89-4aa5-891f-c1c03202ca09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:28.014007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b4e4b-7a89-4aa5-891f-c1c03202ca09 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:28.021081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 446/902] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 22:20:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:28.026802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b4e4b-7a89-4aa5-891f-c1c03202ca09 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:28.033287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:28.066247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e8864-cb11-45ef-9a44-b3a7f72d0d8c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:28.105860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230dc5a-fce5-4531-9c02-749a5becb217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:28.106931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230dc5a-fce5-4531-9c02-749a5becb217 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:28.107469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230dc5a-fce5-4531-9c02-749a5becb217 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:28.107962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230dc5a-fce5-4531-9c02-749a5becb217 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:28.108985 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230dc5a-fce5-4531-9c02-749a5becb217 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:28.109501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230dc5a-fce5-4531-9c02-749a5becb217 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:28.111327 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230dc5a-fce5-4531-9c02-749a5becb217 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:28.137341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 457/903] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 22:20:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:28.138562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230dc5a-fce5-4531-9c02-749a5becb217 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:28.152369 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b4e4b-7a89-4aa5-891f-c1c03202ca09 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:28.186736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fbda0-63e6-41c5-b8a5-6c50a793b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:28.187785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fbda0-63e6-41c5-b8a5-6c50a793b1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:28.187785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fbda0-63e6-41c5-b8a5-6c50a793b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:28.187958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fbda0-63e6-41c5-b8a5-6c50a793b1f7 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:28.188540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fbda0-63e6-41c5-b8a5-6c50a793b1f7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:28.188842 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fbda0-63e6-41c5-b8a5-6c50a793b1f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:28.190189 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fbda0-63e6-41c5-b8a5-6c50a793b1f7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:28.200580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 447/904] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 22:20:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:28.201686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fbda0-63e6-41c5-b8a5-6c50a793b1f7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:28.238216 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:28.275312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:28.286181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230dc5a-fce5-4531-9c02-749a5becb217 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:28.312402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fbda0-63e6-41c5-b8a5-6c50a793b1f7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:28.332928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442d130-b2d2-4b7f-a06e-3e789dee3718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:28.333858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442d130-b2d2-4b7f-a06e-3e789dee3718 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:28.334334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442d130-b2d2-4b7f-a06e-3e789dee3718 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:28.334829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442d130-b2d2-4b7f-a06e-3e789dee3718 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:28.357441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69193f-27ad-4e27-b5d9-9687f0187e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:28.358747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69193f-27ad-4e27-b5d9-9687f0187e81 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:28.359474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69193f-27ad-4e27-b5d9-9687f0187e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:28.359927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69193f-27ad-4e27-b5d9-9687f0187e81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:28.361017 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69193f-27ad-4e27-b5d9-9687f0187e81 None admin] RBAC: Adding query filter params (interface=public, service_id=58a88711d0154c58b1f44acc3a5896c8, region_id=RegionOne) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:28.361816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69193f-27ad-4e27-b5d9-9687f0187e81 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:28.363836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69193f-27ad-4e27-b5d9-9687f0187e81 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:28.375657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 448/905] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 22:20:28 2022] GET /identity/v3/endpoints?service_id=58a88711d0154c58b1f44acc3a5896c8&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:28.376825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69193f-27ad-4e27-b5d9-9687f0187e81 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:28.411831 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442d130-b2d2-4b7f-a06e-3e789dee3718 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:28.413100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442d130-b2d2-4b7f-a06e-3e789dee3718 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:28.496339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69193f-27ad-4e27-b5d9-9687f0187e81 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:28.501555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:28.538112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:28.538311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 458/906] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:28 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:20:28.544083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442d130-b2d2-4b7f-a06e-3e789dee3718 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:28.546786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0053f-200f-44a5-92c0-80a42dd6e3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:28.547567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0053f-200f-44a5-92c0-80a42dd6e3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:28.548866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0053f-200f-44a5-92c0-80a42dd6e3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:28.549295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0053f-200f-44a5-92c0-80a42dd6e3d5 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:28.550080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0053f-200f-44a5-92c0-80a42dd6e3d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:28.551623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0053f-200f-44a5-92c0-80a42dd6e3d5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:28.560249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 449/907] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 22:20:28 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:20:28.657904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442d130-b2d2-4b7f-a06e-3e789dee3718 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:28.700136 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493a8aa-d239-4a13-836d-dc2c52c9a799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:28.700723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493a8aa-d239-4a13-836d-dc2c52c9a799 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:28.701002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493a8aa-d239-4a13-836d-dc2c52c9a799 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:28.701274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493a8aa-d239-4a13-836d-dc2c52c9a799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:28.713797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493a8aa-d239-4a13-836d-dc2c52c9a799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:28.715354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493a8aa-d239-4a13-836d-dc2c52c9a799 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c493a8aa-d239-4a13-836d-dc2c52c9a799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:28.730947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:28.731923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:28.732845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:28.732845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:28.732845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:28.732845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 459/908] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:28.737663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0053f-200f-44a5-92c0-80a42dd6e3d5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:28.846885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0053f-200f-44a5-92c0-80a42dd6e3d5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:28.881804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc92ab-e0de-4e30-994e-fd234bd84019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:28.882409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc92ab-e0de-4e30-994e-fd234bd84019 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:28.882688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc92ab-e0de-4e30-994e-fd234bd84019 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:28.882962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc92ab-e0de-4e30-994e-fd234bd84019 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:28.883603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc92ab-e0de-4e30-994e-fd234bd84019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:28.883905 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc92ab-e0de-4e30-994e-fd234bd84019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:28.885818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc92ab-e0de-4e30-994e-fd234bd84019 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:28.898159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 450/909] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:28.903846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493a8aa-d239-4a13-836d-dc2c52c9a799 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:28.926965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:29.014383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493a8aa-d239-4a13-836d-dc2c52c9a799 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:29.050283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008b76b-85e4-40ae-8c56-081288f2041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:29.050918 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b76b-85e4-40ae-8c56-081288f2041c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:29.051197 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b76b-85e4-40ae-8c56-081288f2041c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:29.051469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b76b-85e4-40ae-8c56-081288f2041c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:29.059125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008b76b-85e4-40ae-8c56-081288f2041c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:29.060682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008b76b-85e4-40ae-8c56-081288f2041c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-0008b76b-85e4-40ae-8c56-081288f2041c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:29.072811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:29.073730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:29.074609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:29.074609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:29.074609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:29.074609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:29.074609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:29.074609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:29.074609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:29.074609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:29.074609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 460/910] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:29.080204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc92ab-e0de-4e30-994e-fd234bd84019 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:29.144408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:29.186435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc92ab-e0de-4e30-994e-fd234bd84019 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:29.221945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb260f9-827b-4336-b9fc-13788b690493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:29.222554 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb260f9-827b-4336-b9fc-13788b690493 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:29.222831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb260f9-827b-4336-b9fc-13788b690493 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:29.223104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb260f9-827b-4336-b9fc-13788b690493 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:29.223793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb260f9-827b-4336-b9fc-13788b690493 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:29.224125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb260f9-827b-4336-b9fc-13788b690493 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:29.225603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb260f9-827b-4336-b9fc-13788b690493 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:29.238123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 451/911] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:29.244811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008b76b-85e4-40ae-8c56-081288f2041c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:29.332255 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:29.351323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008b76b-85e4-40ae-8c56-081288f2041c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:29.388391 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a9ae3-ae2b-47fe-a687-ccb85061b6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:29.389000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a9ae3-ae2b-47fe-a687-ccb85061b6ac None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:29.389278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a9ae3-ae2b-47fe-a687-ccb85061b6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:29.389550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a9ae3-ae2b-47fe-a687-ccb85061b6ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:29.390276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a9ae3-ae2b-47fe-a687-ccb85061b6ac None admin] RBAC: Adding query filter params (role.id=6e6a34bbb7684b249265fcd10027d213, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=77cb38c8c0dc4de3bbdeb611ffd53ff3) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:29.390581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a9ae3-ae2b-47fe-a687-ccb85061b6ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:29.392032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a9ae3-ae2b-47fe-a687-ccb85061b6ac None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:29.398234 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 461/912] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:29 2022] GET /identity/v3/role_assignments?user.id=77cb38c8c0dc4de3bbdeb611ffd53ff3&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=6e6a34bbb7684b249265fcd10027d213 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:29.447461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9d7fa-f441-49b3-a03c-5873b4cc3007 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:29.448151 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9d7fa-f441-49b3-a03c-5873b4cc3007 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:29.448151 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9d7fa-f441-49b3-a03c-5873b4cc3007 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:29.449884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 452/913] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:29.455761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949ad0d-cb77-4b73-babd-da3be316a9ea None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:29.456092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949ad0d-cb77-4b73-babd-da3be316a9ea None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:29.456337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949ad0d-cb77-4b73-babd-da3be316a9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:29.476292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2949ad0d-cb77-4b73-babd-da3be316a9ea None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:29.477527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2949ad0d-cb77-4b73-babd-da3be316a9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:29.555659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 462/914] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:29.578475 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9d7fa-f441-49b3-a03c-5873b4cc3007 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:29.651861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db18ee5-7c14-4529-a282-295af89a11df None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:29.651861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db18ee5-7c14-4529-a282-295af89a11df None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:29.651861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db18ee5-7c14-4529-a282-295af89a11df None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:29.651861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 463/915] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:20:29.660050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69682a-35f2-4402-8bcd-df637c2a1fe8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:29.660050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69682a-35f2-4402-8bcd-df637c2a1fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:29.660050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69682a-35f2-4402-8bcd-df637c2a1fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:29.679357 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3c69682a-35f2-4402-8bcd-df637c2a1fe8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:29.680589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3c69682a-35f2-4402-8bcd-df637c2a1fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:29.689619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9d7fa-f441-49b3-a03c-5873b4cc3007 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:29.708394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:29.727708 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50db045-0a6f-4573-8f61-2781f168cb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:29.729362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50db045-0a6f-4573-8f61-2781f168cb80 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:29.729942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50db045-0a6f-4573-8f61-2781f168cb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:29.730456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50db045-0a6f-4573-8f61-2781f168cb80 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:29.731245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50db045-0a6f-4573-8f61-2781f168cb80 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:29.732898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50db045-0a6f-4573-8f61-2781f168cb80 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-b50db045-0a6f-4573-8f61-2781f168cb80 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:29.745678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:29.746693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:29.747516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:29.752104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 453/916] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 22:20:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:29.754200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50db045-0a6f-4573-8f61-2781f168cb80 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:29.766507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 464/917] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:29.788689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69682a-35f2-4402-8bcd-df637c2a1fe8 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:29.791806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:29.886105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50db045-0a6f-4573-8f61-2781f168cb80 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:29.903201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69682a-35f2-4402-8bcd-df637c2a1fe8 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:29.925596 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a56897-e789-4769-9a01-13f35ba775ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:29.926620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a56897-e789-4769-9a01-13f35ba775ee None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:29.927207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a56897-e789-4769-9a01-13f35ba775ee None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:29.927682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a56897-e789-4769-9a01-13f35ba775ee None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:29.928604 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a56897-e789-4769-9a01-13f35ba775ee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:29.929197 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a56897-e789-4769-9a01-13f35ba775ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:29.930739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a56897-e789-4769-9a01-13f35ba775ee None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:29.940344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 454/918] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 22:20:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:29.949399 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a56897-e789-4769-9a01-13f35ba775ee None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:29.954857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070420a9-c9e4-421a-b025-8b59a1d73dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:29.955557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070420a9-c9e4-421a-b025-8b59a1d73dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:29.955896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070420a9-c9e4-421a-b025-8b59a1d73dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:29.956251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070420a9-c9e4-421a-b025-8b59a1d73dc6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:29.956908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070420a9-c9e4-421a-b025-8b59a1d73dc6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:29.958500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070420a9-c9e4-421a-b025-8b59a1d73dc6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-070420a9-c9e4-421a-b025-8b59a1d73dc6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:29.968717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:29.969720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:29.970564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 465/919] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:20:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:29.978776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070420a9-c9e4-421a-b025-8b59a1d73dc6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:30.003428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:30.060039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:30.072706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a56897-e789-4769-9a01-13f35ba775ee None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:30.096805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070420a9-c9e4-421a-b025-8b59a1d73dc6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:30.120889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88794eb3-937a-4e3d-977a-27dad7a8f6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:30.121742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88794eb3-937a-4e3d-977a-27dad7a8f6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:30.122029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88794eb3-937a-4e3d-977a-27dad7a8f6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:30.122302 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88794eb3-937a-4e3d-977a-27dad7a8f6c9 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:30.123360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88794eb3-937a-4e3d-977a-27dad7a8f6c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:30.123809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88794eb3-937a-4e3d-977a-27dad7a8f6c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:30.125595 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88794eb3-937a-4e3d-977a-27dad7a8f6c9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:30.137891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 455/920] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 22:20:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:30.138860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d9cfb-183f-4797-aeda-63d79d53bfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:30.139109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88794eb3-937a-4e3d-977a-27dad7a8f6c9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:30.139603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d9cfb-183f-4797-aeda-63d79d53bfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:30.140038 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d9cfb-183f-4797-aeda-63d79d53bfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:30.140387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d9cfb-183f-4797-aeda-63d79d53bfb7 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:30.141097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d9cfb-183f-4797-aeda-63d79d53bfb7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:30.141464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d9cfb-183f-4797-aeda-63d79d53bfb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:30.142895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d9cfb-183f-4797-aeda-63d79d53bfb7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:30.152183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 466/921] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:20:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:30.154603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d9cfb-183f-4797-aeda-63d79d53bfb7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:30.262775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d9cfb-183f-4797-aeda-63d79d53bfb7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:30.277130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88794eb3-937a-4e3d-977a-27dad7a8f6c9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:30.289471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:30.295578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:30.307737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a862f-0f5d-4844-bf0f-c2d79ae47950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:30.308373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a862f-0f5d-4844-bf0f-c2d79ae47950 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:30.308647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a862f-0f5d-4844-bf0f-c2d79ae47950 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:30.308917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a862f-0f5d-4844-bf0f-c2d79ae47950 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:30.309544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a862f-0f5d-4844-bf0f-c2d79ae47950 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:30.309844 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a862f-0f5d-4844-bf0f-c2d79ae47950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:30.311198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a862f-0f5d-4844-bf0f-c2d79ae47950 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:30.321238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 467/922] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 22:20:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:30.322646 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a862f-0f5d-4844-bf0f-c2d79ae47950 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:30.330283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aa8ee-edbb-4fd2-8bc3-670f4bdfbb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:30.331141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aa8ee-edbb-4fd2-8bc3-670f4bdfbb3d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:30.332058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aa8ee-edbb-4fd2-8bc3-670f4bdfbb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:30.332442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aa8ee-edbb-4fd2-8bc3-670f4bdfbb3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:30.333254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4aa8ee-edbb-4fd2-8bc3-670f4bdfbb3d None admin] RBAC: Adding query filter params (interface=public, service_id=9d849d721aa8417eb6d52300d803aad4, region_id=RegionOne) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:30.333665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4aa8ee-edbb-4fd2-8bc3-670f4bdfbb3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:30.335583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4aa8ee-edbb-4fd2-8bc3-670f4bdfbb3d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:30.348108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 456/923] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 22:20:30 2022] GET /identity/v3/endpoints?service_id=9d849d721aa8417eb6d52300d803aad4&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:30.350016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aa8ee-edbb-4fd2-8bc3-670f4bdfbb3d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:30.440707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a862f-0f5d-4844-bf0f-c2d79ae47950 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:30.477824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f48da-52aa-4500-80b6-66bed649e5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:30.478500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f48da-52aa-4500-80b6-66bed649e5b6 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:30.478878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f48da-52aa-4500-80b6-66bed649e5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:30.479165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f48da-52aa-4500-80b6-66bed649e5b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:30.479834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f48da-52aa-4500-80b6-66bed649e5b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:30.481484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f48da-52aa-4500-80b6-66bed649e5b6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:30.488681 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aa8ee-edbb-4fd2-8bc3-670f4bdfbb3d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:30.527399 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9421ec-eb18-43bd-b649-d005dfa6d4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:30.528068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9421ec-eb18-43bd-b649-d005dfa6d4cd None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:30.528346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9421ec-eb18-43bd-b649-d005dfa6d4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:30.528669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9421ec-eb18-43bd-b649-d005dfa6d4cd None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:30.529302 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9421ec-eb18-43bd-b649-d005dfa6d4cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:30.530679 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9421ec-eb18-43bd-b649-d005dfa6d4cd None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:30.544111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 457/924] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 22:20:30 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:20:30.557552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 468/925] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:20:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:30.681630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124b080-b270-454c-89f5-ffac953b91d2 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:30.681990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124b080-b270-454c-89f5-ffac953b91d2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:30.682191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124b080-b270-454c-89f5-ffac953b91d2 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:30.683878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 458/926] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:30.689910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd4d8a-8874-426c-8b29-83fc139ce2ca None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:30.690272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd4d8a-8874-426c-8b29-83fc139ce2ca None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:30.690467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd4d8a-8874-426c-8b29-83fc139ce2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:30.710848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-85fd4d8a-8874-426c-8b29-83fc139ce2ca None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:30.711989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-85fd4d8a-8874-426c-8b29-83fc139ce2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:30.767626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:30.793496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 469/927] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:30.818613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124b080-b270-454c-89f5-ffac953b91d2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:30.859622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:30.929125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124b080-b270-454c-89f5-ffac953b91d2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:30.967931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcc9a6-c9ad-4621-9d1b-153ab74b41b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:30.968518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcc9a6-c9ad-4621-9d1b-153ab74b41b5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:30.968937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcc9a6-c9ad-4621-9d1b-153ab74b41b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:30.969207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcc9a6-c9ad-4621-9d1b-153ab74b41b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:30.969803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bcc9a6-c9ad-4621-9d1b-153ab74b41b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:30.971163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bcc9a6-c9ad-4621-9d1b-153ab74b41b5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-07bcc9a6-c9ad-4621-9d1b-153ab74b41b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:30.981301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:30.982280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:30.983179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 459/928] 10.222.0.30 () {56 vars in 1132 bytes} [Sun Aug 7 22:20:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:30.988590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd4d8a-8874-426c-8b29-83fc139ce2ca None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:31.096475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd4d8a-8874-426c-8b29-83fc139ce2ca None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:31.132940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91257005-a5a5-4a15-a341-776c55be481e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:31.133558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91257005-a5a5-4a15-a341-776c55be481e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:31.133840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91257005-a5a5-4a15-a341-776c55be481e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:31.134673 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91257005-a5a5-4a15-a341-776c55be481e None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:31.135316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91257005-a5a5-4a15-a341-776c55be481e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:31.135620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91257005-a5a5-4a15-a341-776c55be481e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:31.137028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91257005-a5a5-4a15-a341-776c55be481e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:31.142300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 470/929] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 22:20:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:31.148136 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcc9a6-c9ad-4621-9d1b-153ab74b41b5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:31.248966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:31.252549 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcc9a6-c9ad-4621-9d1b-153ab74b41b5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:31.293070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454a25a-3d0b-4f6f-aeae-ffbfd684cf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:31.293676 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454a25a-3d0b-4f6f-aeae-ffbfd684cf2f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:31.293956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454a25a-3d0b-4f6f-aeae-ffbfd684cf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:31.294230 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454a25a-3d0b-4f6f-aeae-ffbfd684cf2f None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:31.294860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454a25a-3d0b-4f6f-aeae-ffbfd684cf2f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:31.295164 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454a25a-3d0b-4f6f-aeae-ffbfd684cf2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:31.296553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454a25a-3d0b-4f6f-aeae-ffbfd684cf2f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:31.302278 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 460/930] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 22:20:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:31.308285 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91257005-a5a5-4a15-a341-776c55be481e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:31.355278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:31.418239 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91257005-a5a5-4a15-a341-776c55be481e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:31.453459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167bd2b-8c54-42ea-81e8-a3f3536271bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:31.454113 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167bd2b-8c54-42ea-81e8-a3f3536271bc None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:31.454344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167bd2b-8c54-42ea-81e8-a3f3536271bc None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:31.454606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167bd2b-8c54-42ea-81e8-a3f3536271bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:31.455281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167bd2b-8c54-42ea-81e8-a3f3536271bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:31.456683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167bd2b-8c54-42ea-81e8-a3f3536271bc None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:31.566694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 471/931] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:20:31 2022] POST /identity/v3/endpoints => generated 347 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:31.595625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c678c75-146c-4c7d-a517-6d707b95d794 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:31.596197 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c678c75-146c-4c7d-a517-6d707b95d794 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:31.596197 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c678c75-146c-4c7d-a517-6d707b95d794 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:31.597879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 461/932] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:31.605009 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513be53-93d1-4c48-829d-6c42e5cf5d11 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:31.605697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513be53-93d1-4c48-829d-6c42e5cf5d11 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:31.606178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513be53-93d1-4c48-829d-6c42e5cf5d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:31.628658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f513be53-93d1-4c48-829d-6c42e5cf5d11 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:31.630466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f513be53-93d1-4c48-829d-6c42e5cf5d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:31.717491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 472/933] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:31.741127 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c678c75-146c-4c7d-a517-6d707b95d794 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:31.856276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c678c75-146c-4c7d-a517-6d707b95d794 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:31.892613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df2c27-c567-4a1a-8341-7c1db0256a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:31.893241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df2c27-c567-4a1a-8341-7c1db0256a7e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:31.893524 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df2c27-c567-4a1a-8341-7c1db0256a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:31.893801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df2c27-c567-4a1a-8341-7c1db0256a7e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:31.905401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df2c27-c567-4a1a-8341-7c1db0256a7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f7df2c27-c567-4a1a-8341-7c1db0256a7e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:31.910116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:31.911067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:31.911924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:31.916150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 462/934] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 22:20:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:31.921100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513be53-93d1-4c48-829d-6c42e5cf5d11 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:31.948573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:32.027845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513be53-93d1-4c48-829d-6c42e5cf5d11 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:32.065936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa813c-03dd-49d3-8c74-c981ff11f2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:32.066610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa813c-03dd-49d3-8c74-c981ff11f2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:32.066905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa813c-03dd-49d3-8c74-c981ff11f2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:32.067249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa813c-03dd-49d3-8c74-c981ff11f2b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:32.067969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa813c-03dd-49d3-8c74-c981ff11f2b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:32.068313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa813c-03dd-49d3-8c74-c981ff11f2b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:32.069823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa813c-03dd-49d3-8c74-c981ff11f2b8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:32.079384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 473/935] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 22:20:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:32.085232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df2c27-c567-4a1a-8341-7c1db0256a7e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:32.116580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:32.196469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df2c27-c567-4a1a-8341-7c1db0256a7e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:32.232133 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9d5c5-8d7f-4485-9217-145b3e21ef8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:32.232806 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d5c5-8d7f-4485-9217-145b3e21ef8c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:32.233087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d5c5-8d7f-4485-9217-145b3e21ef8c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:32.233358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d5c5-8d7f-4485-9217-145b3e21ef8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:32.244228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9d5c5-8d7f-4485-9217-145b3e21ef8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:32.245939 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9d5c5-8d7f-4485-9217-145b3e21ef8c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-dfd9d5c5-8d7f-4485-9217-145b3e21ef8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:32.262194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:32.263227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:32.264412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:32.264412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:32.264412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:32.264412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 463/936] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:32.270948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa813c-03dd-49d3-8c74-c981ff11f2b8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:32.363036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:32.379244 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa813c-03dd-49d3-8c74-c981ff11f2b8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:32.416734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9fb28-3665-40b8-a8dc-0810b2d6741d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:32.417551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9fb28-3665-40b8-a8dc-0810b2d6741d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:32.417905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9fb28-3665-40b8-a8dc-0810b2d6741d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:32.418333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9fb28-3665-40b8-a8dc-0810b2d6741d None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:32.419370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9fb28-3665-40b8-a8dc-0810b2d6741d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:32.419682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9fb28-3665-40b8-a8dc-0810b2d6741d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:32.421124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9fb28-3665-40b8-a8dc-0810b2d6741d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:32.434614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 474/937] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:32.440232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9d5c5-8d7f-4485-9217-145b3e21ef8c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:32.521831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:32.546203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9d5c5-8d7f-4485-9217-145b3e21ef8c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:32.582295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049c5b3-adc5-44c4-9d57-26da9543ca80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:32.582931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049c5b3-adc5-44c4-9d57-26da9543ca80 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:32.583209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049c5b3-adc5-44c4-9d57-26da9543ca80 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:32.583484 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049c5b3-adc5-44c4-9d57-26da9543ca80 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:32.592319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049c5b3-adc5-44c4-9d57-26da9543ca80 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:32.593780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049c5b3-adc5-44c4-9d57-26da9543ca80 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f049c5b3-adc5-44c4-9d57-26da9543ca80 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:32.606013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:32.607390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:32.608363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:32.608363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:32.608363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:32.608363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:32.608363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:32.608363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:32.608363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:32.608363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:32.608363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:32.608363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:32.608363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 464/938] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 22:20:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:32.613320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9fb28-3665-40b8-a8dc-0810b2d6741d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:32.720565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847adedd-32e4-4b71-9815-1b15b399e638 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:32.720869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847adedd-32e4-4b71-9815-1b15b399e638 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:32.721028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847adedd-32e4-4b71-9815-1b15b399e638 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:32.724486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9fb28-3665-40b8-a8dc-0810b2d6741d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:32.726876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 465/939] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:20:32.727670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d58530-4174-4cfb-bb59-dc55f4abb33c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:32.728039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d58530-4174-4cfb-bb59-dc55f4abb33c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:32.728326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d58530-4174-4cfb-bb59-dc55f4abb33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:32.750544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-88d58530-4174-4cfb-bb59-dc55f4abb33c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:32.753066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-88d58530-4174-4cfb-bb59-dc55f4abb33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:32.765685 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded51ce-9b76-4619-9ed1-7b169c4436f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:32.766303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded51ce-9b76-4619-9ed1-7b169c4436f2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:32.766580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded51ce-9b76-4619-9ed1-7b169c4436f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:32.766913 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded51ce-9b76-4619-9ed1-7b169c4436f2 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:32.767655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded51ce-9b76-4619-9ed1-7b169c4436f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:32.767966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded51ce-9b76-4619-9ed1-7b169c4436f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:32.769430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded51ce-9b76-4619-9ed1-7b169c4436f2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:32.788768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 475/940] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 22:20:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:32.789875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c90089-6a73-43db-b87c-d62655f93e52 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:32.790163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c90089-6a73-43db-b87c-d62655f93e52 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:32.790552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c90089-6a73-43db-b87c-d62655f93e52 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:32.796087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 476/941] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:32.796087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c90089-6a73-43db-b87c-d62655f93e52 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:32.800663 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:32.848037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 466/942] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:32.848037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07869688-f80a-4a68-a64a-5f5495795328 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:32.848037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07869688-f80a-4a68-a64a-5f5495795328 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:32.848037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07869688-f80a-4a68-a64a-5f5495795328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:32.865567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-07869688-f80a-4a68-a64a-5f5495795328 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:32.866770 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-07869688-f80a-4a68-a64a-5f5495795328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:32.932630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:32.939300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c90089-6a73-43db-b87c-d62655f93e52 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:32.941272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 467/943] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:32.942099 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07869688-f80a-4a68-a64a-5f5495795328 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:32.990323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc66823-ddb4-4d15-8130-2b55c05cffe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:32.990880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc66823-ddb4-4d15-8130-2b55c05cffe6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:32.991159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc66823-ddb4-4d15-8130-2b55c05cffe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:32.991431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc66823-ddb4-4d15-8130-2b55c05cffe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:33.003978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc66823-ddb4-4d15-8130-2b55c05cffe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:33.005465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc66823-ddb4-4d15-8130-2b55c05cffe6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-fdc66823-ddb4-4d15-8130-2b55c05cffe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:33.022316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:33.023429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:33.024367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:33.024367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:33.024367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:33.024367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 477/944] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:33.024605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc66823-ddb4-4d15-8130-2b55c05cffe6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:33.063184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07869688-f80a-4a68-a64a-5f5495795328 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:33.095752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:33.100756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06924f6b-020c-41cf-ae88-762d6add77eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:33.101390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06924f6b-020c-41cf-ae88-762d6add77eb None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:33.101667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06924f6b-020c-41cf-ae88-762d6add77eb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:33.101937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06924f6b-020c-41cf-ae88-762d6add77eb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:33.104115 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06924f6b-020c-41cf-ae88-762d6add77eb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:33.105485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06924f6b-020c-41cf-ae88-762d6add77eb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-06924f6b-020c-41cf-ae88-762d6add77eb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:33.114672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:33.121450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:33.122485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 468/945] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 22:20:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:33.123376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06924f6b-020c-41cf-ae88-762d6add77eb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:33.153453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc66823-ddb4-4d15-8130-2b55c05cffe6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:33.192826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27985cd1-98b3-456b-bef6-b41b4ce0c7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:33.193757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985cd1-98b3-456b-bef6-b41b4ce0c7d8 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:33.194467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985cd1-98b3-456b-bef6-b41b4ce0c7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:33.194928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985cd1-98b3-456b-bef6-b41b4ce0c7d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:33.195939 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27985cd1-98b3-456b-bef6-b41b4ce0c7d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:33.196292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:33.198265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27985cd1-98b3-456b-bef6-b41b4ce0c7d8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:33.202330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: WARNING keystone.server.flask.common [None req-27985cd1-98b3-456b-bef6-b41b4ce0c7d8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 07 22:20:33.246247 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06924f6b-020c-41cf-ae88-762d6add77eb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:33.250208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.sql.core [None req-27985cd1-98b3-456b-bef6-b41b4ce0c7d8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 22:20:33.250208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 07 22:20:33.250208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [parameters: {'id': 'e6426f78a8084856925efa47c096a010', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 07 22:20:33.250208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-27985cd1-98b3-456b-bef6-b41b4ce0c7d8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e6426f78a8084856925efa47c096a010.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e6426f78a8084856925efa47c096a010. Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application conn.query(q) Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 22:20:33.266066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result.read() Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 07 22:20:33.266957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application next(self.gen) Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application yield resource Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application next(self.gen) Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise self.value Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application session.commit() Aug 07 22:20:33.268901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.session.flush() Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application compat.raise_( Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 22:20:33.269805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 22:20:33.270719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application conn.query(q) Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result.read() Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 22:20:33.271522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [parameters: {'id': 'e6426f78a8084856925efa47c096a010', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:33.272479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:33.273620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:33.274768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:33.274768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 07 22:20:33.274768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 07 22:20:33.274768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:33.274768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:33.274768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 07 22:20:33.274768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 07 22:20:33.274768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e6426f78a8084856925efa47c096a010. Aug 07 22:20:33.274768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:33.274768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 478/946] 10.222.0.30 () {60 vars in 1140 bytes} [Sun Aug 7 22:20:33 2022] POST /identity/v3/projects => generated 262 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:33.274768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27985cd1-98b3-456b-bef6-b41b4ce0c7d8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:33.294638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139abd4-2eeb-41eb-8113-6191e6799c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:33.295410 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139abd4-2eeb-41eb-8113-6191e6799c4e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:33.295859 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139abd4-2eeb-41eb-8113-6191e6799c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:33.296219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139abd4-2eeb-41eb-8113-6191e6799c4e None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:33.297280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139abd4-2eeb-41eb-8113-6191e6799c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:33.297785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139abd4-2eeb-41eb-8113-6191e6799c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:33.299591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139abd4-2eeb-41eb-8113-6191e6799c4e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:33.321288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 469/947] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:33.321288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139abd4-2eeb-41eb-8113-6191e6799c4e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:33.392582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27985cd1-98b3-456b-bef6-b41b4ce0c7d8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:33.407789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:33.436062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64736b38-8256-46a5-9f7f-3809c23a2e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:33.436746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64736b38-8256-46a5-9f7f-3809c23a2e6a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:33.437030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64736b38-8256-46a5-9f7f-3809c23a2e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:33.437353 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139abd4-2eeb-41eb-8113-6191e6799c4e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:33.437623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64736b38-8256-46a5-9f7f-3809c23a2e6a None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:33.438000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64736b38-8256-46a5-9f7f-3809c23a2e6a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:33.438714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64736b38-8256-46a5-9f7f-3809c23a2e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:33.441586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64736b38-8256-46a5-9f7f-3809c23a2e6a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:33.456267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 479/948] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 22:20:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:33.456267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64736b38-8256-46a5-9f7f-3809c23a2e6a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:33.459537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:33.476645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed1115-4b57-4c0c-acbb-d69a8311716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:33.477414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed1115-4b57-4c0c-acbb-d69a8311716c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:33.478441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed1115-4b57-4c0c-acbb-d69a8311716c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:33.478914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed1115-4b57-4c0c-acbb-d69a8311716c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:33.487237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed1115-4b57-4c0c-acbb-d69a8311716c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:33.489361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed1115-4b57-4c0c-acbb-d69a8311716c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-12ed1115-4b57-4c0c-acbb-d69a8311716c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:33.503349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:33.504626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:33.505520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:33.505520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:33.505520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:33.505520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:33.505520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:33.505520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:33.505520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:33.505520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:33.505520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 470/949] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:33.505520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed1115-4b57-4c0c-acbb-d69a8311716c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:33.587801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64736b38-8256-46a5-9f7f-3809c23a2e6a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:33.626244 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed1115-4b57-4c0c-acbb-d69a8311716c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:33.635782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cfb34-4891-4b36-a199-fc4a599e076e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:33.637002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cfb34-4891-4b36-a199-fc4a599e076e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:33.637280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cfb34-4891-4b36-a199-fc4a599e076e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:33.637554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cfb34-4891-4b36-a199-fc4a599e076e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:33.644935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cfb34-4891-4b36-a199-fc4a599e076e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:33.646383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cfb34-4891-4b36-a199-fc4a599e076e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-374cfb34-4891-4b36-a199-fc4a599e076e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:33.658635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:33.659742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:33.660691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:33.660691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:33.660691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:33.660691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:33.660691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:33.660691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:33.660691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:33.660691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:33.660691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 480/950] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:33.660691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cfb34-4891-4b36-a199-fc4a599e076e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:33.671454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d52b2d-4b5d-4a42-ba91-532f4cd97837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:33.672136 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d52b2d-4b5d-4a42-ba91-532f4cd97837 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:33.672389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d52b2d-4b5d-4a42-ba91-532f4cd97837 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:33.672659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d52b2d-4b5d-4a42-ba91-532f4cd97837 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:33.673297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d52b2d-4b5d-4a42-ba91-532f4cd97837 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:33.673602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d52b2d-4b5d-4a42-ba91-532f4cd97837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:33.674960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d52b2d-4b5d-4a42-ba91-532f4cd97837 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:33.680988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 471/951] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 22:20:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:33.681781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d52b2d-4b5d-4a42-ba91-532f4cd97837 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:33.705858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:33.741395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:33.776294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cfb34-4891-4b36-a199-fc4a599e076e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:33.804817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d52b2d-4b5d-4a42-ba91-532f4cd97837 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:33.810520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12712b-d7c1-43de-bafc-37c9b768a5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:33.811126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12712b-d7c1-43de-bafc-37c9b768a5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:33.811402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12712b-d7c1-43de-bafc-37c9b768a5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:33.811670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12712b-d7c1-43de-bafc-37c9b768a5f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:33.812400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12712b-d7c1-43de-bafc-37c9b768a5f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:33.812701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12712b-d7c1-43de-bafc-37c9b768a5f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:33.814125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12712b-d7c1-43de-bafc-37c9b768a5f5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:33.840054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 481/952] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:33 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:33.840054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12712b-d7c1-43de-bafc-37c9b768a5f5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:33.864923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdb07f-f89c-430f-8e94-9cd9f7eb4403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:33.865704 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdb07f-f89c-430f-8e94-9cd9f7eb4403 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:33.866004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdb07f-f89c-430f-8e94-9cd9f7eb4403 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:33.866331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdb07f-f89c-430f-8e94-9cd9f7eb4403 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:33.866945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdb07f-f89c-430f-8e94-9cd9f7eb4403 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:33.867249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdb07f-f89c-430f-8e94-9cd9f7eb4403 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:33.869670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdb07f-f89c-430f-8e94-9cd9f7eb4403 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:33.888089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 472/953] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:33.888089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88589c1e-0afc-4364-8138-ff1ac10eb271 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:33.888089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88589c1e-0afc-4364-8138-ff1ac10eb271 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:33.888089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88589c1e-0afc-4364-8138-ff1ac10eb271 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:33.888089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 473/954] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:33.891879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88589c1e-0afc-4364-8138-ff1ac10eb271 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:33.962458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:33.969127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12712b-d7c1-43de-bafc-37c9b768a5f5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:34.010992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88589c1e-0afc-4364-8138-ff1ac10eb271 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:34.013426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:34.016274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442ea30-01d7-4be9-8e78-388e3539e8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:34.016889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442ea30-01d7-4be9-8e78-388e3539e8f2 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:34.017167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442ea30-01d7-4be9-8e78-388e3539e8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:34.017436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442ea30-01d7-4be9-8e78-388e3539e8f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:34.018139 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442ea30-01d7-4be9-8e78-388e3539e8f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:34.019508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442ea30-01d7-4be9-8e78-388e3539e8f2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:34.047043 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4828c-86ad-4cb6-94f3-41cb6b951278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:34.047619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4828c-86ad-4cb6-94f3-41cb6b951278 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:34.047897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4828c-86ad-4cb6-94f3-41cb6b951278 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:34.048209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4828c-86ad-4cb6-94f3-41cb6b951278 None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/77cb38c8c0dc4de3bbdeb611ffd53ff3/roles/6e6a34bbb7684b249265fcd10027d213` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:34.050450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4828c-86ad-4cb6-94f3-41cb6b951278 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=77cb38c8c0dc4de3bbdeb611ffd53ff3, role_id=6e6a34bbb7684b249265fcd10027d213)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:34.051882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4828c-86ad-4cb6-94f3-41cb6b951278 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:34.097142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 482/955] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:20:33 2022] POST /identity/v3/endpoints => generated 349 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:34.098576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b052-d874-4a8a-a017-ceb25e7b4865 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:34.098863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b052-d874-4a8a-a017-ceb25e7b4865 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:34.099138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b052-d874-4a8a-a017-ceb25e7b4865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:34.121216 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8401b052-d874-4a8a-a017-ceb25e7b4865 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:34.124455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8401b052-d874-4a8a-a017-ceb25e7b4865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:34.128228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 474/956] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:20:33 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/77cb38c8c0dc4de3bbdeb611ffd53ff3/roles/6e6a34bbb7684b249265fcd10027d213 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:20:34.220070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 483/957] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:34.243604 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4828c-86ad-4cb6-94f3-41cb6b951278 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:34.366758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4828c-86ad-4cb6-94f3-41cb6b951278 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:34.403448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa8b9d-e3b7-4828-b14b-0a51ca81be8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:34.404098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa8b9d-e3b7-4828-b14b-0a51ca81be8e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:34.404378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa8b9d-e3b7-4828-b14b-0a51ca81be8e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:34.404651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa8b9d-e3b7-4828-b14b-0a51ca81be8e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:34.405237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa8b9d-e3b7-4828-b14b-0a51ca81be8e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:34.406648 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa8b9d-e3b7-4828-b14b-0a51ca81be8e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-95aa8b9d-e3b7-4828-b14b-0a51ca81be8e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:34.415536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:34.416870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:34.418020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 475/958] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:34.423112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401b052-d874-4a8a-a017-ceb25e7b4865 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:34.511275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:34.534644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401b052-d874-4a8a-a017-ceb25e7b4865 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:34.570534 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1e657-15f1-459b-bd35-6bda8fd9bd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:34.571575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1e657-15f1-459b-bd35-6bda8fd9bd63 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:34.572096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1e657-15f1-459b-bd35-6bda8fd9bd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:34.572575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1e657-15f1-459b-bd35-6bda8fd9bd63 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:34.573431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1e657-15f1-459b-bd35-6bda8fd9bd63 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:34.573941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1e657-15f1-459b-bd35-6bda8fd9bd63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:34.575516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1e657-15f1-459b-bd35-6bda8fd9bd63 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:34.581152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 484/959] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:34.592122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa8b9d-e3b7-4828-b14b-0a51ca81be8e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:34.622656 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:34.701703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa8b9d-e3b7-4828-b14b-0a51ca81be8e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:34.737174 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597c960-ed22-4971-a847-d108dd827ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:34.737786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597c960-ed22-4971-a847-d108dd827ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:34.738062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597c960-ed22-4971-a847-d108dd827ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:34.738333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597c960-ed22-4971-a847-d108dd827ce4 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:34.738960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597c960-ed22-4971-a847-d108dd827ce4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:34.739346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597c960-ed22-4971-a847-d108dd827ce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:34.740692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597c960-ed22-4971-a847-d108dd827ce4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:34.745725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 476/960] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:34 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:35.960714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fde95e-c738-4b54-9a18-4f464db25b7b None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:35.961681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fde95e-c738-4b54-9a18-4f464db25b7b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:35.963089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fde95e-c738-4b54-9a18-4f464db25b7b None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:35.964941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 485/961] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:35.970204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e152-350c-423a-a08c-166b3649149a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:35.971009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e152-350c-423a-a08c-166b3649149a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:35.971497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e152-350c-423a-a08c-166b3649149a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:35.996108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9b90e152-350c-423a-a08c-166b3649149a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:35.997618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9b90e152-350c-423a-a08c-166b3649149a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:36.069817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 477/962] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:36.095927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fde95e-c738-4b54-9a18-4f464db25b7b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:36.203916 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7583272-154d-4044-9928-c21ef3e516f6 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.204673 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7583272-154d-4044-9928-c21ef3e516f6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.205289 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7583272-154d-4044-9928-c21ef3e516f6 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.206157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fde95e-c738-4b54-9a18-4f464db25b7b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:36.207388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 478/963] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:36.212909 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3854eaf-f546-4eca-a826-6e05994a418e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.213454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3854eaf-f546-4eca-a826-6e05994a418e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.213940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3854eaf-f546-4eca-a826-6e05994a418e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.232623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c3854eaf-f546-4eca-a826-6e05994a418e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:36.234150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c3854eaf-f546-4eca-a826-6e05994a418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:36.246500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b070b9d-7709-4ac6-b224-af06434c54ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:36.247112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b070b9d-7709-4ac6-b224-af06434c54ed None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.247395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b070b9d-7709-4ac6-b224-af06434c54ed None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.247670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b070b9d-7709-4ac6-b224-af06434c54ed None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.248347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b070b9d-7709-4ac6-b224-af06434c54ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:36.249800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b070b9d-7709-4ac6-b224-af06434c54ed None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:36.254295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:36.308579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 479/964] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:36.310070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd04f4-6061-4711-a935-a8a9adfa5229 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.310359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd04f4-6061-4711-a935-a8a9adfa5229 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.310630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd04f4-6061-4711-a935-a8a9adfa5229 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.311979 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 480/965] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:36.316890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813abb0d-0425-4f91-b9e6-f74acb47b901 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.317176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813abb0d-0425-4f91-b9e6-f74acb47b901 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.317453 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813abb0d-0425-4f91-b9e6-f74acb47b901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.319232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.sql.core [None req-0b070b9d-7709-4ac6-b224-af06434c54ed None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 22:20:36.319232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 07 22:20:36.319232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [parameters: {'user_id': '1cc1467e057645cdbba28ab5be82693e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 07 22:20:36.319232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-0b070b9d-7709-4ac6-b224-af06434c54ed None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application conn.query(q) Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result.read() Aug 07 22:20:36.328065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 22:20:36.329085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application next(self.gen) Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application yield resource Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application next(self.gen) Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise self.value Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application session.commit() Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 22:20:36.330129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.session.flush() Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application compat.raise_( Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 22:20:36.331049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 22:20:36.337788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application conn.query(q) Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result.read() Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 22:20:36.341151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [parameters: {'user_id': '1cc1467e057645cdbba28ab5be82693e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:36.343131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 07 22:20:36.345161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 07 22:20:36.347332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 07 22:20:36.347332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 07 22:20:36.347332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 22:20:36.347332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 22:20:36.347332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 07 22:20:36.347332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:36.347332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 486/966] 10.222.0.30 () {60 vars in 1131 bytes} [Sun Aug 7 22:20:36 2022] POST /identity/v3/users => generated 160 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:36.347332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b070b9d-7709-4ac6-b224-af06434c54ed None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:36.349018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-813abb0d-0425-4f91-b9e6-f74acb47b901 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:36.349181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:36.349299 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-813abb0d-0425-4f91-b9e6-f74acb47b901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:36.418012 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 481/967] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:36.418690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813abb0d-0425-4f91-b9e6-f74acb47b901 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:36.455823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b070b9d-7709-4ac6-b224-af06434c54ed None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:36.490715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461b2bc-eb75-4db8-9d88-22ed79d5effe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:36.491349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461b2bc-eb75-4db8-9d88-22ed79d5effe None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.491626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461b2bc-eb75-4db8-9d88-22ed79d5effe None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.491900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461b2bc-eb75-4db8-9d88-22ed79d5effe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.500211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461b2bc-eb75-4db8-9d88-22ed79d5effe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:36.501905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461b2bc-eb75-4db8-9d88-22ed79d5effe None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:36.505989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-4461b2bc-eb75-4db8-9d88-22ed79d5effe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:36.514746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:36.515802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 487/968] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:20:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:36.517171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461b2bc-eb75-4db8-9d88-22ed79d5effe None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:36.530224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813abb0d-0425-4f91-b9e6-f74acb47b901 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:36.566258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dfd51-c430-4388-8166-8e22a9cd1573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:36.566881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dfd51-c430-4388-8166-8e22a9cd1573 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.567159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dfd51-c430-4388-8166-8e22a9cd1573 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.567435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dfd51-c430-4388-8166-8e22a9cd1573 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.568040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dfd51-c430-4388-8166-8e22a9cd1573 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:36.569396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dfd51-c430-4388-8166-8e22a9cd1573 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d96dfd51-c430-4388-8166-8e22a9cd1573 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:36.579206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:36.580293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:36.581198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 482/969] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 22:20:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:36.582026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dfd51-c430-4388-8166-8e22a9cd1573 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:36.588856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:36.631797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461b2bc-eb75-4db8-9d88-22ed79d5effe None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:36.668144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c0f97-d824-4d67-bf8a-cc5f186855ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:36.671447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c0f97-d824-4d67-bf8a-cc5f186855ea None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.671447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c0f97-d824-4d67-bf8a-cc5f186855ea None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.671447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c0f97-d824-4d67-bf8a-cc5f186855ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.679784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c0f97-d824-4d67-bf8a-cc5f186855ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-361c0f97-d824-4d67-bf8a-cc5f186855ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:36.685472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:36.686435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:36.687298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 488/970] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 22:20:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:36.688651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c0f97-d824-4d67-bf8a-cc5f186855ea None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:36.700969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dfd51-c430-4388-8166-8e22a9cd1573 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:36.737634 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a58dac-759f-4a44-a2cc-db0680ea68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:36.738160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a58dac-759f-4a44-a2cc-db0680ea68d2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.738442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a58dac-759f-4a44-a2cc-db0680ea68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.738719 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a58dac-759f-4a44-a2cc-db0680ea68d2 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.739418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a58dac-759f-4a44-a2cc-db0680ea68d2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:36.739727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a58dac-759f-4a44-a2cc-db0680ea68d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:36.741259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a58dac-759f-4a44-a2cc-db0680ea68d2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:36.777144 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 483/971] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 22:20:36 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:36.777982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a58dac-759f-4a44-a2cc-db0680ea68d2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:36.782085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:36.803092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c0f97-d824-4d67-bf8a-cc5f186855ea None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:36.832690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:36.846658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb630d-cdbf-4742-9b2f-a338f1971302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:36.847313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb630d-cdbf-4742-9b2f-a338f1971302 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.847565 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb630d-cdbf-4742-9b2f-a338f1971302 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.847913 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb630d-cdbf-4742-9b2f-a338f1971302 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.848537 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb630d-cdbf-4742-9b2f-a338f1971302 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:36.848850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb630d-cdbf-4742-9b2f-a338f1971302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:36.850258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb630d-cdbf-4742-9b2f-a338f1971302 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:36.864615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 489/972] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 22:20:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:36.866269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00423312-590a-4bb9-85ad-854d7612042d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.866561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00423312-590a-4bb9-85ad-854d7612042d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.867056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00423312-590a-4bb9-85ad-854d7612042d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.872173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 490/973] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:36.872173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00423312-590a-4bb9-85ad-854d7612042d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:36.894767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a58dac-759f-4a44-a2cc-db0680ea68d2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:36.930724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80157c-6fed-4835-a8af-297a17d193e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:36.931856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80157c-6fed-4835-a8af-297a17d193e2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.931993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80157c-6fed-4835-a8af-297a17d193e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.932298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80157c-6fed-4835-a8af-297a17d193e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.932969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80157c-6fed-4835-a8af-297a17d193e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:36.933277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80157c-6fed-4835-a8af-297a17d193e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:36.935110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80157c-6fed-4835-a8af-297a17d193e2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:36.945456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 484/974] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 22:20:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:36.947016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44feb663-f291-461d-b7f7-585223b09a3a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:36.947341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44feb663-f291-461d-b7f7-585223b09a3a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:36.947567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44feb663-f291-461d-b7f7-585223b09a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:36.971150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-44feb663-f291-461d-b7f7-585223b09a3a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:36.972378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-44feb663-f291-461d-b7f7-585223b09a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:37.002743 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00423312-590a-4bb9-85ad-854d7612042d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:37.038229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:37.043405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e8daa-4992-4332-801d-552f77139686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:37.048819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e8daa-4992-4332-801d-552f77139686 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:37.048819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e8daa-4992-4332-801d-552f77139686 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:37.048819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e8daa-4992-4332-801d-552f77139686 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:37.048819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e8daa-4992-4332-801d-552f77139686 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:37.048819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e8daa-4992-4332-801d-552f77139686 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:37.049253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 485/975] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:37.049347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e8daa-4992-4332-801d-552f77139686 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:37.049431 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44feb663-f291-461d-b7f7-585223b09a3a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:37.052674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 491/976] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 22:20:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:37.058993 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e8daa-4992-4332-801d-552f77139686 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:37.110946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:37.156912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e8daa-4992-4332-801d-552f77139686 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:37.157400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44feb663-f291-461d-b7f7-585223b09a3a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:37.191371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4009c-5247-48e9-8800-7eb7217e099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:37.191610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dd407-daad-4412-8e94-7802b1202e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:37.191975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4009c-5247-48e9-8800-7eb7217e099a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:37.192254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dd407-daad-4412-8e94-7802b1202e41 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:37.192366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4009c-5247-48e9-8800-7eb7217e099a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:37.192535 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dd407-daad-4412-8e94-7802b1202e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:37.192635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4009c-5247-48e9-8800-7eb7217e099a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:37.192812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dd407-daad-4412-8e94-7802b1202e41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:37.193248 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4009c-5247-48e9-8800-7eb7217e099a None admin] RBAC: Adding query filter params (interface=admin, service_id=9d849d721aa8417eb6d52300d803aad4, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:37.193563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4009c-5247-48e9-8800-7eb7217e099a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:37.194927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4009c-5247-48e9-8800-7eb7217e099a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:37.204560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 492/977] 10.222.0.30 () {56 vars in 1249 bytes} [Sun Aug 7 22:20:37 2022] GET /identity/v3/endpoints?service_id=9d849d721aa8417eb6d52300d803aad4&interface=admin®ion_id=RegionOne => generated 193 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:37.204560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4009c-5247-48e9-8800-7eb7217e099a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:37.204815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1dd407-daad-4412-8e94-7802b1202e41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:37.204815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1dd407-daad-4412-8e94-7802b1202e41 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-7d1dd407-daad-4412-8e94-7802b1202e41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:37.219212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:37.220347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:37.221242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:37.221242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:37.221242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:37.221242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 486/978] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:37.221242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dd407-daad-4412-8e94-7802b1202e41 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:37.270068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:37.306336 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4009c-5247-48e9-8800-7eb7217e099a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:37.320837 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dd407-daad-4412-8e94-7802b1202e41 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:37.322329 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:37.339440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fae7e0-3256-42a8-84de-1b58f433addd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:37.340106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fae7e0-3256-42a8-84de-1b58f433addd None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:37.340383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fae7e0-3256-42a8-84de-1b58f433addd None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:37.340657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fae7e0-3256-42a8-84de-1b58f433addd None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:37.341284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fae7e0-3256-42a8-84de-1b58f433addd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:37.342641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fae7e0-3256-42a8-84de-1b58f433addd None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:37.354232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea0ac9-e7fd-4a8c-886b-70d81428e356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:37.354897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea0ac9-e7fd-4a8c-886b-70d81428e356 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:37.355181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea0ac9-e7fd-4a8c-886b-70d81428e356 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:37.357473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea0ac9-e7fd-4a8c-886b-70d81428e356 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:37.357473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea0ac9-e7fd-4a8c-886b-70d81428e356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:37.357473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea0ac9-e7fd-4a8c-886b-70d81428e356 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:37.363314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 487/979] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 22:20:37 2022] GET /identity/v3/services => generated 1497 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:20:37.364113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea0ac9-e7fd-4a8c-886b-70d81428e356 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:37.406178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 493/980] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:20:37 2022] POST /identity/v3/services => generated 242 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:37.488726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea0ac9-e7fd-4a8c-886b-70d81428e356 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:37.504982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:37.538637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531f174-9068-4631-9d3f-ae4c0b4d511f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:37.539179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531f174-9068-4631-9d3f-ae4c0b4d511f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:37.539454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531f174-9068-4631-9d3f-ae4c0b4d511f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:37.539718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531f174-9068-4631-9d3f-ae4c0b4d511f None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:37.540555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531f174-9068-4631-9d3f-ae4c0b4d511f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:37.540859 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531f174-9068-4631-9d3f-ae4c0b4d511f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:37.542345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531f174-9068-4631-9d3f-ae4c0b4d511f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:37.556878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 488/981] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:37.562231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fae7e0-3256-42a8-84de-1b58f433addd None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:37.671157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fae7e0-3256-42a8-84de-1b58f433addd None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:37.706043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2a547-4a70-4b9d-ac4a-6dd2495c6112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:37.706678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2a547-4a70-4b9d-ac4a-6dd2495c6112 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:37.706956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2a547-4a70-4b9d-ac4a-6dd2495c6112 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:37.707390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2a547-4a70-4b9d-ac4a-6dd2495c6112 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:37.715655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2a547-4a70-4b9d-ac4a-6dd2495c6112 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:37.717139 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2a547-4a70-4b9d-ac4a-6dd2495c6112 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-0fc2a547-4a70-4b9d-ac4a-6dd2495c6112 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:37.728406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:37.729508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:37.730729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:37.730729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:37.730729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:37.730729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:37.730729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:37.730729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:37.730729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:37.730729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:37.730729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 22:20:37.730729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:37.730729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 494/982] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 22:20:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:37.735561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531f174-9068-4631-9d3f-ae4c0b4d511f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:37.835030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531f174-9068-4631-9d3f-ae4c0b4d511f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:37.871838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a0f70-854a-44dd-8c56-0986676e1d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:37.872484 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a0f70-854a-44dd-8c56-0986676e1d31 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:37.872761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a0f70-854a-44dd-8c56-0986676e1d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:37.873032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a0f70-854a-44dd-8c56-0986676e1d31 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:37.873818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a0f70-854a-44dd-8c56-0986676e1d31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:37.874174 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a0f70-854a-44dd-8c56-0986676e1d31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:37.875691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a0f70-854a-44dd-8c56-0986676e1d31 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:37.897986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 489/983] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 22:20:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:37.899709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2a547-4a70-4b9d-ac4a-6dd2495c6112 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:37.941930 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:38.013637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2a547-4a70-4b9d-ac4a-6dd2495c6112 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:38.048675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02974258-cda4-48e9-aa0c-fa37136b39ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:38.049278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02974258-cda4-48e9-aa0c-fa37136b39ac None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:38.049555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02974258-cda4-48e9-aa0c-fa37136b39ac None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:38.049824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02974258-cda4-48e9-aa0c-fa37136b39ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:38.117969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02974258-cda4-48e9-aa0c-fa37136b39ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:38.119997 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02974258-cda4-48e9-aa0c-fa37136b39ac None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:38.154005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:38.220191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 495/984] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:37 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 22:20:38.225633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a0f70-854a-44dd-8c56-0986676e1d31 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:38.255019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:38.328359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a0f70-854a-44dd-8c56-0986676e1d31 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:38.361345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc282c-2117-4d97-b331-3c9aaf1e2480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:38.361968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc282c-2117-4d97-b331-3c9aaf1e2480 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:38.362246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc282c-2117-4d97-b331-3c9aaf1e2480 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:38.362519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc282c-2117-4d97-b331-3c9aaf1e2480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:38.373597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc282c-2117-4d97-b331-3c9aaf1e2480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:38.374974 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc282c-2117-4d97-b331-3c9aaf1e2480 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-a2dc282c-2117-4d97-b331-3c9aaf1e2480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:38.389743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:38.390744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:38.396690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:38.396690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:38.396690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:38.396690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 490/985] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:38.396919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02974258-cda4-48e9-aa0c-fa37136b39ac None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:38.501346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02974258-cda4-48e9-aa0c-fa37136b39ac None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:38.537687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846fca0-7509-4332-9e38-9b0feadd27d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:38.538598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846fca0-7509-4332-9e38-9b0feadd27d9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:38.539090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846fca0-7509-4332-9e38-9b0feadd27d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:38.539562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846fca0-7509-4332-9e38-9b0feadd27d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:38.540959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846fca0-7509-4332-9e38-9b0feadd27d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:38.541453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846fca0-7509-4332-9e38-9b0feadd27d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:38.543002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846fca0-7509-4332-9e38-9b0feadd27d9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:38.545749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:38.560236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 496/986] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:38.568397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc282c-2117-4d97-b331-3c9aaf1e2480 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:38.649622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:38.672192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc282c-2117-4d97-b331-3c9aaf1e2480 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:38.708419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd886841-1e76-4436-b637-abc337a32391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:38.709040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd886841-1e76-4436-b637-abc337a32391 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:38.709315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd886841-1e76-4436-b637-abc337a32391 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:38.709581 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd886841-1e76-4436-b637-abc337a32391 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:38.716392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd886841-1e76-4436-b637-abc337a32391 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:38.717988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd886841-1e76-4436-b637-abc337a32391 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-dd886841-1e76-4436-b637-abc337a32391 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:38.729340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:38.730292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:38.736100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:38.736100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:38.736100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:38.736100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:38.736100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:38.736100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:38.736100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:38.736100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:38.736100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 491/987] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:38.736616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846fca0-7509-4332-9e38-9b0feadd27d9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:38.835090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846fca0-7509-4332-9e38-9b0feadd27d9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:38.870190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772d208-8765-4036-9bbd-14ac813b4326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:38.871046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772d208-8765-4036-9bbd-14ac813b4326 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:38.871322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772d208-8765-4036-9bbd-14ac813b4326 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:38.871591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772d208-8765-4036-9bbd-14ac813b4326 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:38.872310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772d208-8765-4036-9bbd-14ac813b4326 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:38.872614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772d208-8765-4036-9bbd-14ac813b4326 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:38.874032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772d208-8765-4036-9bbd-14ac813b4326 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:38.886053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 497/988] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:38.891769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd886841-1e76-4436-b637-abc337a32391 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:39.002726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd886841-1e76-4436-b637-abc337a32391 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:39.036168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b357d-c9ed-49e0-ad27-cd9a98b3ad5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:39.036770 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b357d-c9ed-49e0-ad27-cd9a98b3ad5f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:39.037048 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b357d-c9ed-49e0-ad27-cd9a98b3ad5f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:39.037320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b357d-c9ed-49e0-ad27-cd9a98b3ad5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:39.038039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b357d-c9ed-49e0-ad27-cd9a98b3ad5f None admin] RBAC: Adding query filter params (role.id=6e6a34bbb7684b249265fcd10027d213, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=77cb38c8c0dc4de3bbdeb611ffd53ff3) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:39.038348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b357d-c9ed-49e0-ad27-cd9a98b3ad5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:39.039765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b357d-c9ed-49e0-ad27-cd9a98b3ad5f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:39.052257 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 492/989] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:38 2022] GET /identity/v3/role_assignments?user.id=77cb38c8c0dc4de3bbdeb611ffd53ff3&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=6e6a34bbb7684b249265fcd10027d213 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:39.414102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f84aa-c7cc-4a91-9e13-87160d45a6ad None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:39.414680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f84aa-c7cc-4a91-9e13-87160d45a6ad None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:39.414680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f84aa-c7cc-4a91-9e13-87160d45a6ad None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:39.416704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 498/990] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:39.421767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d3ec6-b6d2-435f-a97c-c864053027a6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:39.422209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d3ec6-b6d2-435f-a97c-c864053027a6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:39.422391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d3ec6-b6d2-435f-a97c-c864053027a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:39.428543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:39.440334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-338d3ec6-b6d2-435f-a97c-c864053027a6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:39.448657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-338d3ec6-b6d2-435f-a97c-c864053027a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:39.478141 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31301878-d0b7-44e8-8693-c012e65d510d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:39.478511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31301878-d0b7-44e8-8693-c012e65d510d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:39.478785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31301878-d0b7-44e8-8693-c012e65d510d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:39.480146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 499/991] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:20:39.485129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7338a-c4e9-4412-9a3a-b51d0c76cd70 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:39.485407 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7338a-c4e9-4412-9a3a-b51d0c76cd70 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:39.485677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7338a-c4e9-4412-9a3a-b51d0c76cd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:39.503955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c8d7338a-c4e9-4412-9a3a-b51d0c76cd70 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:39.505493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c8d7338a-c4e9-4412-9a3a-b51d0c76cd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:39.513213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 493/992] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:39.536901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d3ec6-b6d2-435f-a97c-c864053027a6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:39.570180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:39.589786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 500/993] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:39.611458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7338a-c4e9-4412-9a3a-b51d0c76cd70 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:39.643635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d3ec6-b6d2-435f-a97c-c864053027a6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:39.679391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:39.684181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba687cb-0467-4122-9774-35ec3fdcc2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:39.684816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba687cb-0467-4122-9774-35ec3fdcc2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:39.685095 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba687cb-0467-4122-9774-35ec3fdcc2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:39.685367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba687cb-0467-4122-9774-35ec3fdcc2e3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:39.685955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba687cb-0467-4122-9774-35ec3fdcc2e3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:39.687612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba687cb-0467-4122-9774-35ec3fdcc2e3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-8ba687cb-0467-4122-9774-35ec3fdcc2e3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:39.697457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:39.698516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:39.699371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 494/994] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 22:20:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:39.704652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba687cb-0467-4122-9774-35ec3fdcc2e3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:39.727771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7338a-c4e9-4412-9a3a-b51d0c76cd70 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:39.765143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970edea-7b91-4f54-880a-310e73629828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:39.765795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970edea-7b91-4f54-880a-310e73629828 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:39.766072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970edea-7b91-4f54-880a-310e73629828 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:39.766344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970edea-7b91-4f54-880a-310e73629828 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:39.766937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970edea-7b91-4f54-880a-310e73629828 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:39.768326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970edea-7b91-4f54-880a-310e73629828 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-8970edea-7b91-4f54-880a-310e73629828 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:39.777874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:39.778924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:39.780220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 501/995] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:39.784737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970edea-7b91-4f54-880a-310e73629828 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:39.818649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba687cb-0467-4122-9774-35ec3fdcc2e3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:39.838570 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:39.853794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924ce24-0036-4f56-89ae-43f7f262ffbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:39.854396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924ce24-0036-4f56-89ae-43f7f262ffbd None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:39.854668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924ce24-0036-4f56-89ae-43f7f262ffbd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:39.854938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924ce24-0036-4f56-89ae-43f7f262ffbd None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:39.855561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924ce24-0036-4f56-89ae-43f7f262ffbd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:39.855865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924ce24-0036-4f56-89ae-43f7f262ffbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:39.857266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924ce24-0036-4f56-89ae-43f7f262ffbd None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:39.862748 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 495/996] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 22:20:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:39.869061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924ce24-0036-4f56-89ae-43f7f262ffbd None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:39.894294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970edea-7b91-4f54-880a-310e73629828 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:39.932056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281be58-7515-4786-b23c-d64a8a244302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:39.932498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:39.932625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281be58-7515-4786-b23c-d64a8a244302 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:39.932853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281be58-7515-4786-b23c-d64a8a244302 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:39.933127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281be58-7515-4786-b23c-d64a8a244302 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:39.933756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281be58-7515-4786-b23c-d64a8a244302 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:39.934061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281be58-7515-4786-b23c-d64a8a244302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:39.935440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281be58-7515-4786-b23c-d64a8a244302 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:39.940585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 502/997] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:39.946211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281be58-7515-4786-b23c-d64a8a244302 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:39.985512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924ce24-0036-4f56-89ae-43f7f262ffbd None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:40.024111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d2ea5-36d6-48cc-9033-455202fc27db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:40.025220 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d2ea5-36d6-48cc-9033-455202fc27db None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:40.025702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d2ea5-36d6-48cc-9033-455202fc27db None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:40.026146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d2ea5-36d6-48cc-9033-455202fc27db None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:40.026771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d2ea5-36d6-48cc-9033-455202fc27db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:40.027074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d2ea5-36d6-48cc-9033-455202fc27db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:40.028455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d2ea5-36d6-48cc-9033-455202fc27db None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:40.038110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 496/998] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 22:20:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:40.038932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d2ea5-36d6-48cc-9033-455202fc27db None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:40.057546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281be58-7515-4786-b23c-d64a8a244302 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:40.072609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:40.093988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763259-c03f-42aa-b56f-0d894d0fe03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:40.094817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763259-c03f-42aa-b56f-0d894d0fe03a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:40.095095 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763259-c03f-42aa-b56f-0d894d0fe03a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:40.095369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763259-c03f-42aa-b56f-0d894d0fe03a None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:40.095996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763259-c03f-42aa-b56f-0d894d0fe03a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:40.096328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763259-c03f-42aa-b56f-0d894d0fe03a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:40.097683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763259-c03f-42aa-b56f-0d894d0fe03a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:40.108282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 503/999] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:40.109228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763259-c03f-42aa-b56f-0d894d0fe03a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:40.147398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d2ea5-36d6-48cc-9033-455202fc27db None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:40.172428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:40.184373 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47013e6a-50d2-4e0b-9f89-56cbb2f8827a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:40.185274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47013e6a-50d2-4e0b-9f89-56cbb2f8827a None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:40.185557 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47013e6a-50d2-4e0b-9f89-56cbb2f8827a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:40.185829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47013e6a-50d2-4e0b-9f89-56cbb2f8827a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:40.186517 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47013e6a-50d2-4e0b-9f89-56cbb2f8827a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:40.187919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47013e6a-50d2-4e0b-9f89-56cbb2f8827a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:40.218647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763259-c03f-42aa-b56f-0d894d0fe03a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:40.254575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01df36-25a7-4955-81d0-5a43386ab39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:40.255177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01df36-25a7-4955-81d0-5a43386ab39e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:40.255458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01df36-25a7-4955-81d0-5a43386ab39e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:40.255732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01df36-25a7-4955-81d0-5a43386ab39e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:40.256464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01df36-25a7-4955-81d0-5a43386ab39e None admin] RBAC: Adding query filter params (interface=public, service_id=0cc8f71c4da24e47b9d4bccfcf66df75, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:40.256777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01df36-25a7-4955-81d0-5a43386ab39e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:40.258146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01df36-25a7-4955-81d0-5a43386ab39e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:40.263624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 504/1000] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 22:20:40 2022] GET /identity/v3/endpoints?service_id=0cc8f71c4da24e47b9d4bccfcf66df75&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:40.276157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 497/1001] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:20:40 2022] POST /identity/v3/endpoints => generated 325 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:40.281830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01df36-25a7-4955-81d0-5a43386ab39e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:40.326005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:40.388551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01df36-25a7-4955-81d0-5a43386ab39e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:40.434088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a66793-8ba4-4016-b666-417c7f5fd7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:40.434765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a66793-8ba4-4016-b666-417c7f5fd7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:40.434952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a66793-8ba4-4016-b666-417c7f5fd7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:40.435219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a66793-8ba4-4016-b666-417c7f5fd7e0 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:40.435819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a66793-8ba4-4016-b666-417c7f5fd7e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:40.437279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a66793-8ba4-4016-b666-417c7f5fd7e0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:40.443615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 505/1002] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 22:20:40 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:20:41.092019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e156d-dc2d-475f-abe8-c996673e9376 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:41.092565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e156d-dc2d-475f-abe8-c996673e9376 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:41.092565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e156d-dc2d-475f-abe8-c996673e9376 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:41.094046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 498/1003] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:41.098977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf402b-c5cc-4fa8-9168-2c4d9d3574eb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:41.099263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf402b-c5cc-4fa8-9168-2c4d9d3574eb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:41.099534 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf402b-c5cc-4fa8-9168-2c4d9d3574eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:41.117290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0fdf402b-c5cc-4fa8-9168-2c4d9d3574eb None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:41.119245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0fdf402b-c5cc-4fa8-9168-2c4d9d3574eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:41.189140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 506/1004] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:41 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:41.209960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e156d-dc2d-475f-abe8-c996673e9376 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:41.310545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e156d-dc2d-475f-abe8-c996673e9376 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:41.344099 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224a85f-3ec6-4a9b-b100-afaaef3c7cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:41.344718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224a85f-3ec6-4a9b-b100-afaaef3c7cce None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:41.344995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224a85f-3ec6-4a9b-b100-afaaef3c7cce None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:41.345264 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224a85f-3ec6-4a9b-b100-afaaef3c7cce None admin] PATH_INFO: `/v3/services/image` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:41.345841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224a85f-3ec6-4a9b-b100-afaaef3c7cce None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:41.347185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224a85f-3ec6-4a9b-b100-afaaef3c7cce None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c224a85f-3ec6-4a9b-b100-afaaef3c7cce None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:41.355876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:41.356881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 22:20:41.357718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:41.359089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 499/1005] 10.222.0.30 () {56 vars in 1105 bytes} [Sun Aug 7 22:20:41 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:41.362842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf402b-c5cc-4fa8-9168-2c4d9d3574eb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:41.393735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:41.466942 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf402b-c5cc-4fa8-9168-2c4d9d3574eb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:41.501726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39197690-b042-4cd8-abcf-388200a9372b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:41.502336 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39197690-b042-4cd8-abcf-388200a9372b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:41.502611 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39197690-b042-4cd8-abcf-388200a9372b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:41.502879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39197690-b042-4cd8-abcf-388200a9372b None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:41.503503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39197690-b042-4cd8-abcf-388200a9372b None admin] RBAC: Adding query filter params (name=image) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:41.503805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39197690-b042-4cd8-abcf-388200a9372b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:41.505194 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39197690-b042-4cd8-abcf-388200a9372b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:41.510040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 507/1006] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:41.515421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224a85f-3ec6-4a9b-b100-afaaef3c7cce None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:41.566755 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:41.617721 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224a85f-3ec6-4a9b-b100-afaaef3c7cce None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:41.653795 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b82d29-6f98-44cc-aef6-39a7691f1853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:41.654439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b82d29-6f98-44cc-aef6-39a7691f1853 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:41.654715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b82d29-6f98-44cc-aef6-39a7691f1853 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:41.655027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b82d29-6f98-44cc-aef6-39a7691f1853 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:41.655657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b82d29-6f98-44cc-aef6-39a7691f1853 None admin] RBAC: Adding query filter params (type=image) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:41.655957 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b82d29-6f98-44cc-aef6-39a7691f1853 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:41.657322 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b82d29-6f98-44cc-aef6-39a7691f1853 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:41.661892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 500/1007] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:42.327432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c4843-3485-44ee-86c7-ad68477ad158 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:42.327831 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c4843-3485-44ee-86c7-ad68477ad158 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:42.328008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c4843-3485-44ee-86c7-ad68477ad158 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:42.329514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 508/1008] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:42.334439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadae057-0c21-4991-8738-0b5797da68e5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:42.334872 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadae057-0c21-4991-8738-0b5797da68e5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:42.335000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadae057-0c21-4991-8738-0b5797da68e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:42.353998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-aadae057-0c21-4991-8738-0b5797da68e5 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:42.355217 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-aadae057-0c21-4991-8738-0b5797da68e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:42.423639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 501/1009] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:42 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:42.439646 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61b3e2-55d0-4038-9c5c-d1d7697a34c3 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:42.439950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61b3e2-55d0-4038-9c5c-d1d7697a34c3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:42.440223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61b3e2-55d0-4038-9c5c-d1d7697a34c3 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:42.441690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 509/1010] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:42.446789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61b3e2-55d0-4038-9c5c-d1d7697a34c3 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:42.447289 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cb6b6-9bec-4177-97d0-293e65aa824f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:42.447493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cb6b6-9bec-4177-97d0-293e65aa824f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:42.447767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cb6b6-9bec-4177-97d0-293e65aa824f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:42.465216 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-308cb6b6-9bec-4177-97d0-293e65aa824f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:42.466389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-308cb6b6-9bec-4177-97d0-293e65aa824f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:42.502291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:42.534466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 502/1011] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:42 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:42.545343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:42.552637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61b3e2-55d0-4038-9c5c-d1d7697a34c3 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:42.557047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cb6b6-9bec-4177-97d0-293e65aa824f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:42.587728 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27492ce-8f23-422d-b826-015b77bebb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:42.588366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27492ce-8f23-422d-b826-015b77bebb2d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:42.588845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27492ce-8f23-422d-b826-015b77bebb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:42.589528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27492ce-8f23-422d-b826-015b77bebb2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:42.590633 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27492ce-8f23-422d-b826-015b77bebb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:42.592162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27492ce-8f23-422d-b826-015b77bebb2d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:42.593722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 510/1012] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:42.599567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27492ce-8f23-422d-b826-015b77bebb2d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:42.661899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cb6b6-9bec-4177-97d0-293e65aa824f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:42.697396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe19f9a-4839-4c95-846a-838cf58490b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:42.698290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe19f9a-4839-4c95-846a-838cf58490b1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:42.698773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe19f9a-4839-4c95-846a-838cf58490b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:42.699219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe19f9a-4839-4c95-846a-838cf58490b1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:42.699955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe19f9a-4839-4c95-846a-838cf58490b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:42.701499 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe19f9a-4839-4c95-846a-838cf58490b1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:42.701687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27492ce-8f23-422d-b826-015b77bebb2d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-afe19f9a-4839-4c95-846a-838cf58490b1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:42.710689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:42.712462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 22:20:42.713800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:42.716182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 503/1013] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:42.721783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe19f9a-4839-4c95-846a-838cf58490b1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:42.744951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce7343-bcff-4301-a300-624caa3d2057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:42.745824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce7343-bcff-4301-a300-624caa3d2057 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:42.746094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce7343-bcff-4301-a300-624caa3d2057 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:42.746363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce7343-bcff-4301-a300-624caa3d2057 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:42.746990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce7343-bcff-4301-a300-624caa3d2057 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:42.748566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce7343-bcff-4301-a300-624caa3d2057 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:42.762784 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:42.818299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 511/1014] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:20:42 2022] POST /identity/v3/projects => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:42.835504 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe19f9a-4839-4c95-846a-838cf58490b1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:42.875825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3434608-bd7d-4d07-8078-417b0f952f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:42.876459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3434608-bd7d-4d07-8078-417b0f952f57 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:42.876750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3434608-bd7d-4d07-8078-417b0f952f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:42.877025 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3434608-bd7d-4d07-8078-417b0f952f57 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:42.877685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3434608-bd7d-4d07-8078-417b0f952f57 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:42.877992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3434608-bd7d-4d07-8078-417b0f952f57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:42.879406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3434608-bd7d-4d07-8078-417b0f952f57 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:42.885746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 504/1015] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:42.891403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce7343-bcff-4301-a300-624caa3d2057 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:42.912554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:43.006538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce7343-bcff-4301-a300-624caa3d2057 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:43.043491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd21ff-41ae-4512-bbcc-1103d587d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:43.044132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd21ff-41ae-4512-bbcc-1103d587d8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:43.044415 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd21ff-41ae-4512-bbcc-1103d587d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:43.044688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd21ff-41ae-4512-bbcc-1103d587d8f2 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:43.045317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd21ff-41ae-4512-bbcc-1103d587d8f2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:43.045620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd21ff-41ae-4512-bbcc-1103d587d8f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:43.046987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd21ff-41ae-4512-bbcc-1103d587d8f2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:43.051997 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 512/1016] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 22:20:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:43.057585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3434608-bd7d-4d07-8078-417b0f952f57 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:43.156110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:43.159643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3434608-bd7d-4d07-8078-417b0f952f57 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:43.194368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcc7a8-ed5f-42c3-bb1d-d8c6a1337e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:43.194970 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc7a8-ed5f-42c3-bb1d-d8c6a1337e1b None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:43.195247 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc7a8-ed5f-42c3-bb1d-d8c6a1337e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:43.195515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc7a8-ed5f-42c3-bb1d-d8c6a1337e1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:43.196922 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcc7a8-ed5f-42c3-bb1d-d8c6a1337e1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:43.198100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcc7a8-ed5f-42c3-bb1d-d8c6a1337e1b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:43.275165 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 505/1017] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:20:43 2022] POST /identity/v3/endpoints => generated 334 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:43.709111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193388d-0c68-422d-8722-c9091739ef35 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:43.709635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193388d-0c68-422d-8722-c9091739ef35 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:43.709635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193388d-0c68-422d-8722-c9091739ef35 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:43.711003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 513/1018] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:43.717652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528650-d405-4d62-93f5-b328c02c833e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:43.717652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528650-d405-4d62-93f5-b328c02c833e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:43.717652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528650-d405-4d62-93f5-b328c02c833e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:43.736449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-91528650-d405-4d62-93f5-b328c02c833e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:43.737646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-91528650-d405-4d62-93f5-b328c02c833e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:43.811715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 506/1019] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:43 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:43.835291 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193388d-0c68-422d-8722-c9091739ef35 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:43.917894 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:43.941453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193388d-0c68-422d-8722-c9091739ef35 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:43.983472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b095c5-867e-4289-bf65-7940867b3683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:43.984047 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b095c5-867e-4289-bf65-7940867b3683 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:43.984338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b095c5-867e-4289-bf65-7940867b3683 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:43.984614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b095c5-867e-4289-bf65-7940867b3683 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:43.985297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b095c5-867e-4289-bf65-7940867b3683 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:43.986716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b095c5-867e-4289-bf65-7940867b3683 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:44.088040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 514/1020] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:20:43 2022] POST /identity/v3/services => generated 242 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:44.881964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38943c-fc6d-4cde-8aeb-5defa1303e97 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:44.882519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38943c-fc6d-4cde-8aeb-5defa1303e97 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:44.882519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38943c-fc6d-4cde-8aeb-5defa1303e97 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:44.884219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 507/1021] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:44.889062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e53472-269c-4839-a3a5-3a45f4eb97c0 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:44.889467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e53472-269c-4839-a3a5-3a45f4eb97c0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:44.889650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e53472-269c-4839-a3a5-3a45f4eb97c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:44.907925 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-82e53472-269c-4839-a3a5-3a45f4eb97c0 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:44.909133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-82e53472-269c-4839-a3a5-3a45f4eb97c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:44.989537 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 515/1022] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:45.014626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38943c-fc6d-4cde-8aeb-5defa1303e97 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:45.097591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:45.116533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38943c-fc6d-4cde-8aeb-5defa1303e97 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:45.152097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298253b-2bb0-4071-9828-4116e1e67fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:45.152656 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298253b-2bb0-4071-9828-4116e1e67fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:45.152933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298253b-2bb0-4071-9828-4116e1e67fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:45.153205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298253b-2bb0-4071-9828-4116e1e67fa1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:45.154316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298253b-2bb0-4071-9828-4116e1e67fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:45.155735 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298253b-2bb0-4071-9828-4116e1e67fa1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:45.157476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 508/1023] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:45.162920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e53472-269c-4839-a3a5-3a45f4eb97c0 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:45.230365 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:45.265452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e53472-269c-4839-a3a5-3a45f4eb97c0 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:45.299101 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f042f-9752-496e-8b33-bf3698566331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:45.299701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f042f-9752-496e-8b33-bf3698566331 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:45.299976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f042f-9752-496e-8b33-bf3698566331 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:45.300293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f042f-9752-496e-8b33-bf3698566331 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:45.300970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f042f-9752-496e-8b33-bf3698566331 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:45.302387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f042f-9752-496e-8b33-bf3698566331 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:45.392919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615069b-4bdc-4ff1-a263-6f884b20e1d8 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:45.393202 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615069b-4bdc-4ff1-a263-6f884b20e1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:45.393507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615069b-4bdc-4ff1-a263-6f884b20e1d8 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:45.394995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 509/1024] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:45.408133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 516/1025] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:20:45 2022] POST /identity/v3/users => generated 285 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:45.410494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d09588-37ef-41d6-83f6-3b64f55ed63d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:45.410778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d09588-37ef-41d6-83f6-3b64f55ed63d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:45.411053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d09588-37ef-41d6-83f6-3b64f55ed63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:45.432649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e1d09588-37ef-41d6-83f6-3b64f55ed63d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:45.433879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e1d09588-37ef-41d6-83f6-3b64f55ed63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:45.510493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 510/1026] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:45.534732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f042f-9752-496e-8b33-bf3698566331 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:45.639753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f042f-9752-496e-8b33-bf3698566331 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:45.663954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:45.675179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9cbca-a210-467e-b485-b8d205555c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:45.675876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9cbca-a210-467e-b485-b8d205555c4f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:45.676324 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9cbca-a210-467e-b485-b8d205555c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:45.676606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9cbca-a210-467e-b485-b8d205555c4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:45.688232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9cbca-a210-467e-b485-b8d205555c4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-7db9cbca-a210-467e-b485-b8d205555c4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:45.692819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:45.693770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:45.694956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 517/1027] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 22:20:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:45.700511 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d09588-37ef-41d6-83f6-3b64f55ed63d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:45.784455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:45.803692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d09588-37ef-41d6-83f6-3b64f55ed63d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:45.840137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517b06d-de42-41ff-be85-0ae988a5f3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:45.840739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517b06d-de42-41ff-be85-0ae988a5f3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:45.841015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517b06d-de42-41ff-be85-0ae988a5f3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:45.841285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517b06d-de42-41ff-be85-0ae988a5f3b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:45.841936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517b06d-de42-41ff-be85-0ae988a5f3b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:45.842237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517b06d-de42-41ff-be85-0ae988a5f3b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:45.843603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517b06d-de42-41ff-be85-0ae988a5f3b0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:45.851444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 511/1028] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 22:20:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:45.857393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9cbca-a210-467e-b485-b8d205555c4f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:45.963998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9cbca-a210-467e-b485-b8d205555c4f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:46.002760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91a01b-0ee5-463b-b8ea-b892a74ec784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:46.003447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91a01b-0ee5-463b-b8ea-b892a74ec784 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:46.003729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91a01b-0ee5-463b-b8ea-b892a74ec784 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:46.004061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91a01b-0ee5-463b-b8ea-b892a74ec784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:46.014845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91a01b-0ee5-463b-b8ea-b892a74ec784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:46.016359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91a01b-0ee5-463b-b8ea-b892a74ec784 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-af91a01b-0ee5-463b-b8ea-b892a74ec784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:46.030384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:46.031359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:46.032387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:46.032387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:46.032387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:46.036201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 518/1029] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:46.037397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517b06d-de42-41ff-be85-0ae988a5f3b0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:46.110067 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd20bb-2073-4deb-8327-00563f6cb638 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:46.110341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd20bb-2073-4deb-8327-00563f6cb638 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:46.110613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd20bb-2073-4deb-8327-00563f6cb638 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:46.112100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 519/1030] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:46.116819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fe810-6dc8-4de5-a482-5431a47a10ef None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:46.117099 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fe810-6dc8-4de5-a482-5431a47a10ef None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:46.117369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fe810-6dc8-4de5-a482-5431a47a10ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:46.136712 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-432fe810-6dc8-4de5-a482-5431a47a10ef None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:46.137903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-432fe810-6dc8-4de5-a482-5431a47a10ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:46.144862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517b06d-de42-41ff-be85-0ae988a5f3b0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:46.165502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:46.181824 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52caf577-12ce-4f7a-a585-59797db1df1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:46.182427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52caf577-12ce-4f7a-a585-59797db1df1d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:46.182718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52caf577-12ce-4f7a-a585-59797db1df1d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:46.182979 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52caf577-12ce-4f7a-a585-59797db1df1d None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:46.183619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52caf577-12ce-4f7a-a585-59797db1df1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:46.183920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52caf577-12ce-4f7a-a585-59797db1df1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:46.185304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52caf577-12ce-4f7a-a585-59797db1df1d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:46.201923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 512/1031] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:46.203309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52caf577-12ce-4f7a-a585-59797db1df1d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:46.206236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:46.212700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 520/1032] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:46.235505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fe810-6dc8-4de5-a482-5431a47a10ef None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:46.310237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52caf577-12ce-4f7a-a585-59797db1df1d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:46.339873 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fe810-6dc8-4de5-a482-5431a47a10ef None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:46.346362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4631e-7307-473d-92c9-51f5374c3420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:46.346979 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4631e-7307-473d-92c9-51f5374c3420 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:46.347257 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4631e-7307-473d-92c9-51f5374c3420 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:46.347527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4631e-7307-473d-92c9-51f5374c3420 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:46.356122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4631e-7307-473d-92c9-51f5374c3420 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:46.357554 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4631e-7307-473d-92c9-51f5374c3420 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-58d4631e-7307-473d-92c9-51f5374c3420 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:46.369186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:46.370328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:46.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:46.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:46.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:46.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:46.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:46.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:46.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:46.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:46.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:46.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:46.371578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 513/1033] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 22:20:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:46.376084 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4631e-7307-473d-92c9-51f5374c3420 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:46.381343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337ace0-7c51-4f54-a0f1-df33122a1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:46.381974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337ace0-7c51-4f54-a0f1-df33122a1c88 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:46.382250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337ace0-7c51-4f54-a0f1-df33122a1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:46.382520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337ace0-7c51-4f54-a0f1-df33122a1c88 None admin] PATH_INFO: `/v3/services/image` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:46.383104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337ace0-7c51-4f54-a0f1-df33122a1c88 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:46.385161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337ace0-7c51-4f54-a0f1-df33122a1c88 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-1337ace0-7c51-4f54-a0f1-df33122a1c88 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:46.393908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:46.395084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:46.396129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 521/1034] 10.222.0.30 () {56 vars in 1105 bytes} [Sun Aug 7 22:20:46 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:46.401111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337ace0-7c51-4f54-a0f1-df33122a1c88 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:46.431174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:46.448162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:46.488437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4631e-7307-473d-92c9-51f5374c3420 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:46.515645 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337ace0-7c51-4f54-a0f1-df33122a1c88 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:46.525552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941be93-4650-4309-8f4a-de0904e800f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:46.526169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941be93-4650-4309-8f4a-de0904e800f2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:46.526438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941be93-4650-4309-8f4a-de0904e800f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:46.527034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941be93-4650-4309-8f4a-de0904e800f2 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:46.527763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941be93-4650-4309-8f4a-de0904e800f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:46.528100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941be93-4650-4309-8f4a-de0904e800f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:46.529520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941be93-4650-4309-8f4a-de0904e800f2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:46.552762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 514/1035] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 22:20:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:46.553766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941be93-4650-4309-8f4a-de0904e800f2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:46.556627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c16d4-f6d4-415b-bbd8-db7c079bc0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:46.557233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c16d4-f6d4-415b-bbd8-db7c079bc0fe None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:46.557507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c16d4-f6d4-415b-bbd8-db7c079bc0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:46.558522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c16d4-f6d4-415b-bbd8-db7c079bc0fe None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:46.558522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c16d4-f6d4-415b-bbd8-db7c079bc0fe None admin] RBAC: Adding query filter params (name=image) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:46.558703 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c16d4-f6d4-415b-bbd8-db7c079bc0fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:46.560041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c16d4-f6d4-415b-bbd8-db7c079bc0fe None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:46.565376 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 522/1036] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:46.570744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c16d4-f6d4-415b-bbd8-db7c079bc0fe None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:46.668627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941be93-4650-4309-8f4a-de0904e800f2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:46.668894 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:46.679030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c16d4-f6d4-415b-bbd8-db7c079bc0fe None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:46.691171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:46.706408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953f8a0-a887-42c9-88c2-33c407c2637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:46.707017 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953f8a0-a887-42c9-88c2-33c407c2637e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:46.707303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953f8a0-a887-42c9-88c2-33c407c2637e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:46.707594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953f8a0-a887-42c9-88c2-33c407c2637e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:46.719057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48a22b-6f5d-45e1-a9f0-23d96ca8ddae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:46.719906 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48a22b-6f5d-45e1-a9f0-23d96ca8ddae None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:46.720390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48a22b-6f5d-45e1-a9f0-23d96ca8ddae None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:46.720856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48a22b-6f5d-45e1-a9f0-23d96ca8ddae None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:46.721557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48a22b-6f5d-45e1-a9f0-23d96ca8ddae None admin] RBAC: Adding query filter params (type=image) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:46.721860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48a22b-6f5d-45e1-a9f0-23d96ca8ddae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:46.723224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48a22b-6f5d-45e1-a9f0-23d96ca8ddae None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:46.728471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 523/1037] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:46.733811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48a22b-6f5d-45e1-a9f0-23d96ca8ddae None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:46.794055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953f8a0-a887-42c9-88c2-33c407c2637e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:46.795519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953f8a0-a887-42c9-88c2-33c407c2637e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:46.842460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48a22b-6f5d-45e1-a9f0-23d96ca8ddae None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:46.876475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75844fb0-9e97-4c33-a7fa-03206665c9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:46.877077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75844fb0-9e97-4c33-a7fa-03206665c9fa None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:46.877363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75844fb0-9e97-4c33-a7fa-03206665c9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:46.877636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75844fb0-9e97-4c33-a7fa-03206665c9fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:46.878362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75844fb0-9e97-4c33-a7fa-03206665c9fa None admin] RBAC: Adding query filter params (interface=public, service_id=a31c1d7e853f4abd912933697d4cdfc0, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:46.878674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75844fb0-9e97-4c33-a7fa-03206665c9fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:46.880081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75844fb0-9e97-4c33-a7fa-03206665c9fa None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:46.886439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 524/1038] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 22:20:46 2022] GET /identity/v3/endpoints?service_id=a31c1d7e853f4abd912933697d4cdfc0&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:46.892153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75844fb0-9e97-4c33-a7fa-03206665c9fa None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:46.909241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 515/1039] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:20:46.909828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953f8a0-a887-42c9-88c2-33c407c2637e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:46.917363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:46.931659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:47.002562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75844fb0-9e97-4c33-a7fa-03206665c9fa None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:47.025465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953f8a0-a887-42c9-88c2-33c407c2637e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:47.041721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e434e-8fae-423b-b3b7-8902c4ee74ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:47.042332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e434e-8fae-423b-b3b7-8902c4ee74ec None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:47.042609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e434e-8fae-423b-b3b7-8902c4ee74ec None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:47.042892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e434e-8fae-423b-b3b7-8902c4ee74ec None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:47.043487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e434e-8fae-423b-b3b7-8902c4ee74ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:47.044962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e434e-8fae-423b-b3b7-8902c4ee74ec None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:47.051697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 525/1040] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 22:20:46 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:20:47.062611 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d7466-931a-4d6f-a6c6-001b7bdd5a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:47.063316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d7466-931a-4d6f-a6c6-001b7bdd5a19 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:47.063540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d7466-931a-4d6f-a6c6-001b7bdd5a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:47.063809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d7466-931a-4d6f-a6c6-001b7bdd5a19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:47.076809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d7466-931a-4d6f-a6c6-001b7bdd5a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:47.078290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d7466-931a-4d6f-a6c6-001b7bdd5a19 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c91d7466-931a-4d6f-a6c6-001b7bdd5a19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:47.095746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:47.096884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:47.097797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:47.097797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:47.097797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:47.097797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 516/1041] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:47.103381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e434e-8fae-423b-b3b7-8902c4ee74ec None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:47.196283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:47.214500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e434e-8fae-423b-b3b7-8902c4ee74ec None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:47.250428 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb027ec1-7574-40af-86a8-d56571b4bed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:47.251040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb027ec1-7574-40af-86a8-d56571b4bed1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:47.251318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb027ec1-7574-40af-86a8-d56571b4bed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:47.251590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb027ec1-7574-40af-86a8-d56571b4bed1 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:47.252265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb027ec1-7574-40af-86a8-d56571b4bed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:47.252572 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb027ec1-7574-40af-86a8-d56571b4bed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:47.253940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb027ec1-7574-40af-86a8-d56571b4bed1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:47.266512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 526/1042] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:47.272247 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d7466-931a-4d6f-a6c6-001b7bdd5a19 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:47.345199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:47.376221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d7466-931a-4d6f-a6c6-001b7bdd5a19 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:47.410332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca7950-e70d-42f1-8ede-38962b0c038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:47.410952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca7950-e70d-42f1-8ede-38962b0c038c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:47.411227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca7950-e70d-42f1-8ede-38962b0c038c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:47.411495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca7950-e70d-42f1-8ede-38962b0c038c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:47.418357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca7950-e70d-42f1-8ede-38962b0c038c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:47.419809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca7950-e70d-42f1-8ede-38962b0c038c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-02ca7950-e70d-42f1-8ede-38962b0c038c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:47.430889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:47.431779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:47.433061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:47.433061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:47.433061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:47.433061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:47.433061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:47.433061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:47.433061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:47.433061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:47.436079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 517/1043] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:47.437833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb027ec1-7574-40af-86a8-d56571b4bed1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:47.511961 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ac025-d4fa-4b19-8bee-d85b51c74743 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:47.512352 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ac025-d4fa-4b19-8bee-d85b51c74743 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:47.512570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ac025-d4fa-4b19-8bee-d85b51c74743 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:47.514068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 518/1044] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:20:47.519444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46564cde-d610-4e71-ab22-8faf14b9b639 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:47.520240 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46564cde-d610-4e71-ab22-8faf14b9b639 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:47.520522 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46564cde-d610-4e71-ab22-8faf14b9b639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:47.541967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-46564cde-d610-4e71-ab22-8faf14b9b639 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:47.542181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb027ec1-7574-40af-86a8-d56571b4bed1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:47.543162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-46564cde-d610-4e71-ab22-8faf14b9b639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:47.576554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b17414-84f5-4fbd-bcc7-6454f65c6f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:47.577156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b17414-84f5-4fbd-bcc7-6454f65c6f10 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:47.577433 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b17414-84f5-4fbd-bcc7-6454f65c6f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:47.577700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b17414-84f5-4fbd-bcc7-6454f65c6f10 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:47.578385 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b17414-84f5-4fbd-bcc7-6454f65c6f10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:47.578687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b17414-84f5-4fbd-bcc7-6454f65c6f10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:47.580132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b17414-84f5-4fbd-bcc7-6454f65c6f10 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:47.592307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 527/1045] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:47.598179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b17414-84f5-4fbd-bcc7-6454f65c6f10 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:47.601769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:47.636064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 519/1046] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:47.653863 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46564cde-d610-4e71-ab22-8faf14b9b639 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:47.705024 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b17414-84f5-4fbd-bcc7-6454f65c6f10 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:47.716176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:47.744177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a3fd2-f6cb-4a26-8653-2c13733ea2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:47.745096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a3fd2-f6cb-4a26-8653-2c13733ea2ba None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:47.745549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a3fd2-f6cb-4a26-8653-2c13733ea2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:47.746219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a3fd2-f6cb-4a26-8653-2c13733ea2ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:47.747096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a3fd2-f6cb-4a26-8653-2c13733ea2ba None admin] RBAC: Adding query filter params (role.id=6e6a34bbb7684b249265fcd10027d213, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=8effbd0670ef41f59b46feb9f0997554) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:47.747553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a3fd2-f6cb-4a26-8653-2c13733ea2ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:47.750223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a3fd2-f6cb-4a26-8653-2c13733ea2ba None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:47.756529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 528/1047] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:47 2022] GET /identity/v3/role_assignments?user.id=8effbd0670ef41f59b46feb9f0997554&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=6e6a34bbb7684b249265fcd10027d213 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:47.771764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46564cde-d610-4e71-ab22-8faf14b9b639 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:47.811834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43fd32-c481-4003-82e7-d683650c54a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:47.812527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43fd32-c481-4003-82e7-d683650c54a8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:47.812812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43fd32-c481-4003-82e7-d683650c54a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:47.813086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43fd32-c481-4003-82e7-d683650c54a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:47.825813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43fd32-c481-4003-82e7-d683650c54a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-cd43fd32-c481-4003-82e7-d683650c54a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:47.830687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:47.831643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:47.832778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 520/1048] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:47.838525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a3fd2-f6cb-4a26-8653-2c13733ea2ba None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:47.913328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:47.950514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a3fd2-f6cb-4a26-8653-2c13733ea2ba None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:47.987180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc2f71-c8df-4c5d-81e1-2f4093de4423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:47.987789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc2f71-c8df-4c5d-81e1-2f4093de4423 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:47.988093 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc2f71-c8df-4c5d-81e1-2f4093de4423 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:47.988369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc2f71-c8df-4c5d-81e1-2f4093de4423 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:47.989030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc2f71-c8df-4c5d-81e1-2f4093de4423 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:47.989340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc2f71-c8df-4c5d-81e1-2f4093de4423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:47.990714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc2f71-c8df-4c5d-81e1-2f4093de4423 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:48.004950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 529/1049] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:20:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:48.006619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43fd32-c481-4003-82e7-d683650c54a8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:48.113834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43fd32-c481-4003-82e7-d683650c54a8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:48.137887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:48.149622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d367601-9af0-43f9-bee7-859bae54801b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:48.150472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d367601-9af0-43f9-bee7-859bae54801b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:48.150643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d367601-9af0-43f9-bee7-859bae54801b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:48.151043 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d367601-9af0-43f9-bee7-859bae54801b None admin] PATH_INFO: `/v3/users/46259818b9714181b24407fc6b9f0e07` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:48.164674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d367601-9af0-43f9-bee7-859bae54801b None admin] RBAC: Authorizing `identity:get_user(user_id=46259818b9714181b24407fc6b9f0e07)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:48.166732 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d367601-9af0-43f9-bee7-859bae54801b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:48.169623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 521/1050] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:20:48 2022] GET /identity/v3/users/46259818b9714181b24407fc6b9f0e07 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:48.174345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc2f71-c8df-4c5d-81e1-2f4093de4423 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:48.279627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc2f71-c8df-4c5d-81e1-2f4093de4423 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:48.307940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:48.313830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f76ade-7697-4464-88f0-a18170e30667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:48.314429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f76ade-7697-4464-88f0-a18170e30667 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:48.314704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f76ade-7697-4464-88f0-a18170e30667 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:48.314975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f76ade-7697-4464-88f0-a18170e30667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:48.382475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f76ade-7697-4464-88f0-a18170e30667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:48.383957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f76ade-7697-4464-88f0-a18170e30667 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:48.480439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 530/1051] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:48 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 22:20:48.486001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d367601-9af0-43f9-bee7-859bae54801b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:48.586459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d367601-9af0-43f9-bee7-859bae54801b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:48.620974 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f6e20-84c0-4971-840e-36a0fd23d014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:48.621594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f6e20-84c0-4971-840e-36a0fd23d014 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:48.621870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f6e20-84c0-4971-840e-36a0fd23d014 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:48.622139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f6e20-84c0-4971-840e-36a0fd23d014 None admin] PATH_INFO: `/v3/projects/029a0727429443f491e126247ff12b68` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:48.623251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f6e20-84c0-4971-840e-36a0fd23d014 None admin] RBAC: Authorizing `identity:get_project(project_id=029a0727429443f491e126247ff12b68)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:48.624692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f6e20-84c0-4971-840e-36a0fd23d014 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:48.626301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 522/1052] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:20:48 2022] GET /identity/v3/projects/029a0727429443f491e126247ff12b68 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:48.632340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f76ade-7697-4464-88f0-a18170e30667 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:48.663049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:48.738552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f76ade-7697-4464-88f0-a18170e30667 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:48.772005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d610b42-2102-42c8-9bd6-212367c45f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:48.772625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d610b42-2102-42c8-9bd6-212367c45f7f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:48.772902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d610b42-2102-42c8-9bd6-212367c45f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:48.773173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d610b42-2102-42c8-9bd6-212367c45f7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:48.773904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d610b42-2102-42c8-9bd6-212367c45f7f None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=029a0727429443f491e126247ff12b68, user.id=46259818b9714181b24407fc6b9f0e07) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:48.774820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d610b42-2102-42c8-9bd6-212367c45f7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:48.776479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d610b42-2102-42c8-9bd6-212367c45f7f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:48.787651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 531/1053] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:48 2022] GET /identity/v3/role_assignments?user.id=46259818b9714181b24407fc6b9f0e07&scope.project.id=029a0727429443f491e126247ff12b68&role.id=108664305d2546949f7327b65d4cb667 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:49.086180 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c926c-07cd-4c9c-99f7-7837fe433244 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:49.086540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c926c-07cd-4c9c-99f7-7837fe433244 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:49.087291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c926c-07cd-4c9c-99f7-7837fe433244 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:49.088701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 523/1054] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:49.093796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5f6cd-20ba-4cf8-b8e9-8832980eb124 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:49.094279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5f6cd-20ba-4cf8-b8e9-8832980eb124 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:49.094361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5f6cd-20ba-4cf8-b8e9-8832980eb124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:49.113638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3cd5f6cd-20ba-4cf8-b8e9-8832980eb124 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:49.114853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3cd5f6cd-20ba-4cf8-b8e9-8832980eb124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:49.186023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 532/1055] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:49.208502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c926c-07cd-4c9c-99f7-7837fe433244 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:49.260260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:49.312619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c926c-07cd-4c9c-99f7-7837fe433244 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:49.346045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227953d-4558-453d-8295-2849222fcdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:49.346585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227953d-4558-453d-8295-2849222fcdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:49.346859 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227953d-4558-453d-8295-2849222fcdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:49.347126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227953d-4558-453d-8295-2849222fcdf8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:49.347699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227953d-4558-453d-8295-2849222fcdf8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:49.349156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227953d-4558-453d-8295-2849222fcdf8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f227953d-4558-453d-8295-2849222fcdf8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:49.358158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:49.359138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 22:20:49.360440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:49.364283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 524/1056] 10.222.0.30 () {56 vars in 1105 bytes} [Sun Aug 7 22:20:49 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:49.365116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5f6cd-20ba-4cf8-b8e9-8832980eb124 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:49.390297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:49.466263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5f6cd-20ba-4cf8-b8e9-8832980eb124 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:49.500704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf4a07-bd85-45a3-b657-27f5ecf975ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:49.501307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf4a07-bd85-45a3-b657-27f5ecf975ca None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:49.501586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf4a07-bd85-45a3-b657-27f5ecf975ca None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:49.501855 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf4a07-bd85-45a3-b657-27f5ecf975ca None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:49.502478 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf4a07-bd85-45a3-b657-27f5ecf975ca None admin] RBAC: Adding query filter params (name=image) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:49.502777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf4a07-bd85-45a3-b657-27f5ecf975ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:49.504156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf4a07-bd85-45a3-b657-27f5ecf975ca None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:49.509866 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 533/1057] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:49.515237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227953d-4558-453d-8295-2849222fcdf8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:49.619703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227953d-4558-453d-8295-2849222fcdf8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:49.633253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:49.654422 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484851f-955c-4cc9-8d1c-a038747ca9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:49.655227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484851f-955c-4cc9-8d1c-a038747ca9cf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:49.655678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484851f-955c-4cc9-8d1c-a038747ca9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:49.656122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484851f-955c-4cc9-8d1c-a038747ca9cf None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:49.656890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484851f-955c-4cc9-8d1c-a038747ca9cf None admin] RBAC: Adding query filter params (type=image) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:49.657341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484851f-955c-4cc9-8d1c-a038747ca9cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:49.658869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484851f-955c-4cc9-8d1c-a038747ca9cf None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:49.669589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 525/1058] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:49.670839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf4a07-bd85-45a3-b657-27f5ecf975ca None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:49.761117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:49.765490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe4fe0-d922-4fab-af52-e8c8aff88979 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:49.765777 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe4fe0-d922-4fab-af52-e8c8aff88979 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:49.766056 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe4fe0-d922-4fab-af52-e8c8aff88979 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:49.767447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 526/1059] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:49.772059 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7833-57f8-4be6-a2f1-5df28972b443 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:49.772348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7833-57f8-4be6-a2f1-5df28972b443 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:49.772621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7833-57f8-4be6-a2f1-5df28972b443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:49.775119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf4a07-bd85-45a3-b657-27f5ecf975ca None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:49.791635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-162f7833-57f8-4be6-a2f1-5df28972b443 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:49.792877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-162f7833-57f8-4be6-a2f1-5df28972b443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:49.811825 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf648f3-8149-45fb-9791-66a53f084b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:49.812458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf648f3-8149-45fb-9791-66a53f084b18 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:49.812733 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf648f3-8149-45fb-9791-66a53f084b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:49.813001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf648f3-8149-45fb-9791-66a53f084b18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:49.813664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf648f3-8149-45fb-9791-66a53f084b18 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:49.815010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf648f3-8149-45fb-9791-66a53f084b18 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:49.861804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 527/1060] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:49.888143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 534/1061] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:20:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:49.889734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f7833-57f8-4be6-a2f1-5df28972b443 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:50.008407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f7833-57f8-4be6-a2f1-5df28972b443 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:50.018343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:50.054497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df441b49-d6b9-4f7c-8972-7c8b782acd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:50.055123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df441b49-d6b9-4f7c-8972-7c8b782acd9c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:50.056050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df441b49-d6b9-4f7c-8972-7c8b782acd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:50.056275 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df441b49-d6b9-4f7c-8972-7c8b782acd9c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:50.066587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df441b49-d6b9-4f7c-8972-7c8b782acd9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-df441b49-d6b9-4f7c-8972-7c8b782acd9c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:50.071462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:50.072550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:50.073400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 528/1062] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 22:20:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:50.078864 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf648f3-8149-45fb-9791-66a53f084b18 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:50.183843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf648f3-8149-45fb-9791-66a53f084b18 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:50.217861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8462f25-13d9-41e8-a578-8edaaf2a631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:50.218472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8462f25-13d9-41e8-a578-8edaaf2a631a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:50.218746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8462f25-13d9-41e8-a578-8edaaf2a631a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:50.219013 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8462f25-13d9-41e8-a578-8edaaf2a631a None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:50.219674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8462f25-13d9-41e8-a578-8edaaf2a631a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:50.219972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8462f25-13d9-41e8-a578-8edaaf2a631a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:50.221356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8462f25-13d9-41e8-a578-8edaaf2a631a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:50.234467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 535/1063] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 22:20:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:50.236071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df441b49-d6b9-4f7c-8972-7c8b782acd9c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:50.339592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df441b49-d6b9-4f7c-8972-7c8b782acd9c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:50.376300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbea1c-882e-419d-8a75-da1e9384c33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:50.376935 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbea1c-882e-419d-8a75-da1e9384c33a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:50.377218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbea1c-882e-419d-8a75-da1e9384c33a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:50.377493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbea1c-882e-419d-8a75-da1e9384c33a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:50.388543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbea1c-882e-419d-8a75-da1e9384c33a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:50.389983 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbea1c-882e-419d-8a75-da1e9384c33a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-adbbea1c-882e-419d-8a75-da1e9384c33a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:50.404901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:50.405896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:50.406907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:50.406907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:50.406907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:50.408328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 529/1064] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:50.411834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8462f25-13d9-41e8-a578-8edaaf2a631a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:50.421991 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:50.515134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8462f25-13d9-41e8-a578-8edaaf2a631a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:50.548971 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617a019-19e9-4769-b119-2250f21d34c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:50.549580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617a019-19e9-4769-b119-2250f21d34c2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:50.549865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617a019-19e9-4769-b119-2250f21d34c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:50.550140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617a019-19e9-4769-b119-2250f21d34c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:50.550783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617a019-19e9-4769-b119-2250f21d34c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:50.551090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617a019-19e9-4769-b119-2250f21d34c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:50.552488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617a019-19e9-4769-b119-2250f21d34c2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:50.565304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 536/1065] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:50.570825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbea1c-882e-419d-8a75-da1e9384c33a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:50.587578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:50.677135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbea1c-882e-419d-8a75-da1e9384c33a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:50.711153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1956fe-33bb-499c-ae21-e21a7b5a0312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:50.711940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1956fe-33bb-499c-ae21-e21a7b5a0312 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:50.712420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1956fe-33bb-499c-ae21-e21a7b5a0312 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:50.712703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1956fe-33bb-499c-ae21-e21a7b5a0312 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:50.720113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1956fe-33bb-499c-ae21-e21a7b5a0312 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:50.721532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1956fe-33bb-499c-ae21-e21a7b5a0312 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-3b1956fe-33bb-499c-ae21-e21a7b5a0312 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:50.732941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:50.733888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:50.734831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:50.734831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:50.734831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:50.734831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:50.734831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:50.734831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:50.734831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:50.734831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:50.734831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:50.734831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:50.736343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 530/1066] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 22:20:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:50.739886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617a019-19e9-4769-b119-2250f21d34c2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:50.792995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7eb84-015a-46f4-8569-55d26d51ab05 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:50.793290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7eb84-015a-46f4-8569-55d26d51ab05 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:50.793462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7eb84-015a-46f4-8569-55d26d51ab05 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:50.794930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 531/1067] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:20:50.799911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c1352-a957-4356-bdef-725f472978a3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:50.800228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c1352-a957-4356-bdef-725f472978a3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:50.800501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c1352-a957-4356-bdef-725f472978a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:50.807841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:50.819332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b68c1352-a957-4356-bdef-725f472978a3 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:50.821759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b68c1352-a957-4356-bdef-725f472978a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:50.844230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617a019-19e9-4769-b119-2250f21d34c2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:50.881928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f29ae8-bd2f-4ec1-a293-07c0a08ab2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:50.882544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f29ae8-bd2f-4ec1-a293-07c0a08ab2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:50.882831 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f29ae8-bd2f-4ec1-a293-07c0a08ab2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:50.883109 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f29ae8-bd2f-4ec1-a293-07c0a08ab2a4 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:50.883936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f29ae8-bd2f-4ec1-a293-07c0a08ab2a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:50.884291 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f29ae8-bd2f-4ec1-a293-07c0a08ab2a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:50.885708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f29ae8-bd2f-4ec1-a293-07c0a08ab2a4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:50.893299 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 532/1068] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:50.904229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 537/1069] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 22:20:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:50.909801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c1352-a957-4356-bdef-725f472978a3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:50.913270 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:50.916631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f29ae8-bd2f-4ec1-a293-07c0a08ab2a4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:51.027020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f29ae8-bd2f-4ec1-a293-07c0a08ab2a4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:51.032055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c1352-a957-4356-bdef-725f472978a3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:51.053643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:51.064796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7284fbd-2967-4cc7-ad9f-b98f2e8dc8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:51.065419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7284fbd-2967-4cc7-ad9f-b98f2e8dc8cd None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.065695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7284fbd-2967-4cc7-ad9f-b98f2e8dc8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.065962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7284fbd-2967-4cc7-ad9f-b98f2e8dc8cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.075420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968958-8476-49c9-af6d-83c130ca0454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:51.076478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968958-8476-49c9-af6d-83c130ca0454 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.076614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7284fbd-2967-4cc7-ad9f-b98f2e8dc8cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:51.087470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968958-8476-49c9-af6d-83c130ca0454 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.087470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968958-8476-49c9-af6d-83c130ca0454 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f7284fbd-2967-4cc7-ad9f-b98f2e8dc8cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:51.087730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:51.088739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 538/1070] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:51.089590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7284fbd-2967-4cc7-ad9f-b98f2e8dc8cd None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:51.091437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8968958-8476-49c9-af6d-83c130ca0454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:51.092919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8968958-8476-49c9-af6d-83c130ca0454 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-e8968958-8476-49c9-af6d-83c130ca0454 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:51.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:51.109516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:51.110407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:51.110407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:51.110407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:51.110407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 533/1071] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:51.115494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968958-8476-49c9-af6d-83c130ca0454 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:51.171260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:51.205454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7284fbd-2967-4cc7-ad9f-b98f2e8dc8cd None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:51.218431 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968958-8476-49c9-af6d-83c130ca0454 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:51.244092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937930e-b4cb-49c0-820d-4f0476ebf67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:51.244695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937930e-b4cb-49c0-820d-4f0476ebf67c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.244970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937930e-b4cb-49c0-820d-4f0476ebf67c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.245239 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937930e-b4cb-49c0-820d-4f0476ebf67c None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.246138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937930e-b4cb-49c0-820d-4f0476ebf67c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:51.246441 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937930e-b4cb-49c0-820d-4f0476ebf67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:51.247863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937930e-b4cb-49c0-820d-4f0476ebf67c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:51.255203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881818f-1852-46ce-a6c4-9131f6441b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:51.255800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881818f-1852-46ce-a6c4-9131f6441b28 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.256349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881818f-1852-46ce-a6c4-9131f6441b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.256474 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 539/1072] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:20:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:51.256554 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881818f-1852-46ce-a6c4-9131f6441b28 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.256990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881818f-1852-46ce-a6c4-9131f6441b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:51.257346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881818f-1852-46ce-a6c4-9131f6441b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:51.259404 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881818f-1852-46ce-a6c4-9131f6441b28 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:51.262520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937930e-b4cb-49c0-820d-4f0476ebf67c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:51.280518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 534/1073] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:51.281545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881818f-1852-46ce-a6c4-9131f6441b28 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:51.311652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:51.367359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937930e-b4cb-49c0-820d-4f0476ebf67c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:51.395007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881818f-1852-46ce-a6c4-9131f6441b28 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:51.403112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7978d1b-4877-48d7-8c51-0a6aecdcb945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:51.403817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7978d1b-4877-48d7-8c51-0a6aecdcb945 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.404188 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7978d1b-4877-48d7-8c51-0a6aecdcb945 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.404476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7978d1b-4877-48d7-8c51-0a6aecdcb945 None admin] PATH_INFO: `/v3/users/46259818b9714181b24407fc6b9f0e07` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.405615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7978d1b-4877-48d7-8c51-0a6aecdcb945 None admin] RBAC: Authorizing `identity:get_user(user_id=46259818b9714181b24407fc6b9f0e07)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:51.407046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7978d1b-4877-48d7-8c51-0a6aecdcb945 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:51.408657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 540/1074] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:20:51 2022] GET /identity/v3/users/46259818b9714181b24407fc6b9f0e07 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:51.414147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:51.416487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7978d1b-4877-48d7-8c51-0a6aecdcb945 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:51.432973 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6abdf0-4f82-4669-ad51-49177583f106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:51.433860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6abdf0-4f82-4669-ad51-49177583f106 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.433972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6abdf0-4f82-4669-ad51-49177583f106 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.434187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6abdf0-4f82-4669-ad51-49177583f106 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.440950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6abdf0-4f82-4669-ad51-49177583f106 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:51.442370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6abdf0-4f82-4669-ad51-49177583f106 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-9e6abdf0-4f82-4669-ad51-49177583f106 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:51.454537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:51.455435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:51.456389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:51.456389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:51.456389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:51.456389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:51.456389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:51.456389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:51.456389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:51.456389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:51.456389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 535/1075] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:51.461391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6abdf0-4f82-4669-ad51-49177583f106 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:51.533422 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7978d1b-4877-48d7-8c51-0a6aecdcb945 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:51.548454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:51.568695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182ed78-4d03-4334-9af3-b987a657e9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:51.569240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182ed78-4d03-4334-9af3-b987a657e9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.569513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182ed78-4d03-4334-9af3-b987a657e9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.569782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182ed78-4d03-4334-9af3-b987a657e9c5 None admin] PATH_INFO: `/v3/projects/029a0727429443f491e126247ff12b68` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.570911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182ed78-4d03-4334-9af3-b987a657e9c5 None admin] RBAC: Authorizing `identity:get_project(project_id=029a0727429443f491e126247ff12b68)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:51.572787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182ed78-4d03-4334-9af3-b987a657e9c5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:51.573483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6abdf0-4f82-4669-ad51-49177583f106 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:51.574649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 541/1076] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:20:51 2022] GET /identity/v3/projects/029a0727429443f491e126247ff12b68 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:51.580552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182ed78-4d03-4334-9af3-b987a657e9c5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:51.612637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4534936-00b9-4270-adc8-0d871946ba30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:51.613298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4534936-00b9-4270-adc8-0d871946ba30 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.613578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4534936-00b9-4270-adc8-0d871946ba30 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.613900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4534936-00b9-4270-adc8-0d871946ba30 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.614592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4534936-00b9-4270-adc8-0d871946ba30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:51.614900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4534936-00b9-4270-adc8-0d871946ba30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:51.616400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4534936-00b9-4270-adc8-0d871946ba30 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:51.629559 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 536/1077] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:51.635705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4534936-00b9-4270-adc8-0d871946ba30 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:51.671776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:51.688868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182ed78-4d03-4334-9af3-b987a657e9c5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:51.726046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a797c7-76c9-43db-be08-1b9da767315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:51.726705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a797c7-76c9-43db-be08-1b9da767315a None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.726989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a797c7-76c9-43db-be08-1b9da767315a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.727899 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a797c7-76c9-43db-be08-1b9da767315a None admin] PATH_INFO: `/v3/projects/029a0727429443f491e126247ff12b68/users/46259818b9714181b24407fc6b9f0e07/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.729961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a797c7-76c9-43db-be08-1b9da767315a None admin] RBAC: Authorizing `identity:create_grant(project_id=029a0727429443f491e126247ff12b68, user_id=46259818b9714181b24407fc6b9f0e07, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:51.731399 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a797c7-76c9-43db-be08-1b9da767315a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:51.741761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4534936-00b9-4270-adc8-0d871946ba30 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:51.777387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347af69-1c72-4201-a1bc-9a70290fad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:51.778034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347af69-1c72-4201-a1bc-9a70290fad46 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.778308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347af69-1c72-4201-a1bc-9a70290fad46 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.778635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347af69-1c72-4201-a1bc-9a70290fad46 None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/8effbd0670ef41f59b46feb9f0997554/roles/6e6a34bbb7684b249265fcd10027d213` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.781814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347af69-1c72-4201-a1bc-9a70290fad46 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=8effbd0670ef41f59b46feb9f0997554, role_id=6e6a34bbb7684b249265fcd10027d213)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:51.783255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347af69-1c72-4201-a1bc-9a70290fad46 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:51.811266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 542/1078] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:20:51 2022] PUT /identity/v3/projects/029a0727429443f491e126247ff12b68/users/46259818b9714181b24407fc6b9f0e07/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:20:51.897427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 537/1079] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:20:51 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/8effbd0670ef41f59b46feb9f0997554/roles/6e6a34bbb7684b249265fcd10027d213 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:20:51.932393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edf4ad-7281-4425-ae1c-a370568e59dc None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.932720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edf4ad-7281-4425-ae1c-a370568e59dc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.932950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edf4ad-7281-4425-ae1c-a370568e59dc None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.934767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 543/1080] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:20:51.940768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba5c8a-1ded-45ee-86be-76a8c4766ecb None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:51.940992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba5c8a-1ded-45ee-86be-76a8c4766ecb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:51.941272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba5c8a-1ded-45ee-86be-76a8c4766ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:51.964215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-42ba5c8a-1ded-45ee-86be-76a8c4766ecb None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:51.965506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-42ba5c8a-1ded-45ee-86be-76a8c4766ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:52.043307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 538/1081] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:52.065592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edf4ad-7281-4425-ae1c-a370568e59dc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:52.114199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:52.172217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edf4ad-7281-4425-ae1c-a370568e59dc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:52.208536 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3bced-4553-4a8f-8100-85c3ca0c7351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:52.209302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3bced-4553-4a8f-8100-85c3ca0c7351 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:52.209495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3bced-4553-4a8f-8100-85c3ca0c7351 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:52.209770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3bced-4553-4a8f-8100-85c3ca0c7351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:52.280262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3bced-4553-4a8f-8100-85c3ca0c7351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:52.281737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3bced-4553-4a8f-8100-85c3ca0c7351 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:52.324229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:52.383255 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 544/1082] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:20:52.389269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba5c8a-1ded-45ee-86be-76a8c4766ecb None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:52.417901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:52.490959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba5c8a-1ded-45ee-86be-76a8c4766ecb None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:52.525627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1927f65-dd49-41a6-bbd3-0f9037fcd5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:52.526250 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1927f65-dd49-41a6-bbd3-0f9037fcd5eb None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:52.526525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1927f65-dd49-41a6-bbd3-0f9037fcd5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:52.526796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1927f65-dd49-41a6-bbd3-0f9037fcd5eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:52.536838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1927f65-dd49-41a6-bbd3-0f9037fcd5eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:52.538261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1927f65-dd49-41a6-bbd3-0f9037fcd5eb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d1927f65-dd49-41a6-bbd3-0f9037fcd5eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:52.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:52.553981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:52.560447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:52.560447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:52.560447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:52.560447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 539/1083] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:52.561045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3bced-4553-4a8f-8100-85c3ca0c7351 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:52.663620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3bced-4553-4a8f-8100-85c3ca0c7351 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:52.698793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cb84c-7013-45fd-9ef8-15467fee017d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:52.699406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cb84c-7013-45fd-9ef8-15467fee017d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:52.699743 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cb84c-7013-45fd-9ef8-15467fee017d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:52.699959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cb84c-7013-45fd-9ef8-15467fee017d None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:52.700629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cb84c-7013-45fd-9ef8-15467fee017d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:52.700936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cb84c-7013-45fd-9ef8-15467fee017d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:52.702302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cb84c-7013-45fd-9ef8-15467fee017d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:52.713695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 545/1084] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:53.852630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55d45-b4b9-4cb2-b336-e2e54475aab7 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:53.853163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55d45-b4b9-4cb2-b336-e2e54475aab7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:53.853163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55d45-b4b9-4cb2-b336-e2e54475aab7 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:53.854519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 540/1085] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:53.859644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfabab-d3d4-4cca-a8dc-0c35bc8e49dd None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:53.860114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfabab-d3d4-4cca-a8dc-0c35bc8e49dd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:53.860211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfabab-d3d4-4cca-a8dc-0c35bc8e49dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:53.866457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:53.879179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-acdfabab-d3d4-4cca-a8dc-0c35bc8e49dd None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:53.880421 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-acdfabab-d3d4-4cca-a8dc-0c35bc8e49dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:53.900510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24313ac-e939-411a-861f-a268ff5a7477 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:53.900796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24313ac-e939-411a-861f-a268ff5a7477 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:53.901071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24313ac-e939-411a-861f-a268ff5a7477 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:53.902520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 541/1086] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:20:53.907086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0a258-331c-422c-a3eb-cc6d18810091 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:53.907367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0a258-331c-422c-a3eb-cc6d18810091 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:53.907642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0a258-331c-422c-a3eb-cc6d18810091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:53.926028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f6e0a258-331c-422c-a3eb-cc6d18810091 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:53.927207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f6e0a258-331c-422c-a3eb-cc6d18810091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:53.947373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 546/1087] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:53.970717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfabab-d3d4-4cca-a8dc-0c35bc8e49dd None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.000052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 542/1088] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:20:54.021460 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0a258-331c-422c-a3eb-cc6d18810091 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.031175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:54.069300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfabab-d3d4-4cca-a8dc-0c35bc8e49dd None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:54.099220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:54.104987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2d12e-51c1-4250-88e9-b43fc984a4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:54.105555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d12e-51c1-4250-88e9-b43fc984a4be None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:54.105833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d12e-51c1-4250-88e9-b43fc984a4be None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:54.106111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d12e-51c1-4250-88e9-b43fc984a4be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:54.116162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2d12e-51c1-4250-88e9-b43fc984a4be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:54.120441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0a258-331c-422c-a3eb-cc6d18810091 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-0ad2d12e-51c1-4250-88e9-b43fc984a4be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:54.121794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:54.122889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:54.123729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 547/1089] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:20:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:54.128877 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2d12e-51c1-4250-88e9-b43fc984a4be None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.154584 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0836801-b33d-4737-aaf3-aa668c9ad24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:54.155207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0836801-b33d-4737-aaf3-aa668c9ad24f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:54.155485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0836801-b33d-4737-aaf3-aa668c9ad24f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:54.155757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0836801-b33d-4737-aaf3-aa668c9ad24f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:54.166456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0836801-b33d-4737-aaf3-aa668c9ad24f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-e0836801-b33d-4737-aaf3-aa668c9ad24f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:20:54.171065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:54.172181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:54.173032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 543/1090] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:54.178064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0836801-b33d-4737-aaf3-aa668c9ad24f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.231181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2d12e-51c1-4250-88e9-b43fc984a4be None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:54.245104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:54.265351 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badee252-4325-4401-9991-96396bf4ffd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:54.265969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badee252-4325-4401-9991-96396bf4ffd6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:54.266245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badee252-4325-4401-9991-96396bf4ffd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:54.266513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badee252-4325-4401-9991-96396bf4ffd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:54.267171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badee252-4325-4401-9991-96396bf4ffd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:54.267473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badee252-4325-4401-9991-96396bf4ffd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:54.268861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badee252-4325-4401-9991-96396bf4ffd6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:54.277024 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 548/1091] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:54.282193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0836801-b33d-4737-aaf3-aa668c9ad24f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:54.282833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badee252-4325-4401-9991-96396bf4ffd6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.312475 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3ad1f-8e72-4f5c-b1cf-90cbe0d0fe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:54.313076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3ad1f-8e72-4f5c-b1cf-90cbe0d0fe89 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:54.313354 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3ad1f-8e72-4f5c-b1cf-90cbe0d0fe89 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:54.313630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3ad1f-8e72-4f5c-b1cf-90cbe0d0fe89 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:54.314277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3ad1f-8e72-4f5c-b1cf-90cbe0d0fe89 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:54.314594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3ad1f-8e72-4f5c-b1cf-90cbe0d0fe89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:54.315943 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3ad1f-8e72-4f5c-b1cf-90cbe0d0fe89 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:54.323631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 544/1092] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:54.329145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3ad1f-8e72-4f5c-b1cf-90cbe0d0fe89 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.338167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:54.383037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badee252-4325-4401-9991-96396bf4ffd6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:54.417056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc2465-354a-48d0-8fd1-83fc9a6ed0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:54.417675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc2465-354a-48d0-8fd1-83fc9a6ed0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:54.417951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc2465-354a-48d0-8fd1-83fc9a6ed0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:54.418227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc2465-354a-48d0-8fd1-83fc9a6ed0b6 None admin] PATH_INFO: `/v3/users/46259818b9714181b24407fc6b9f0e07` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:54.419336 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc2465-354a-48d0-8fd1-83fc9a6ed0b6 None admin] RBAC: Authorizing `identity:get_user(user_id=46259818b9714181b24407fc6b9f0e07)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:54.420781 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc2465-354a-48d0-8fd1-83fc9a6ed0b6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:54.422345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 549/1093] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/users/46259818b9714181b24407fc6b9f0e07 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:54.425785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3ad1f-8e72-4f5c-b1cf-90cbe0d0fe89 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:54.428074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc2465-354a-48d0-8fd1-83fc9a6ed0b6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.453659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:54.459349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa651da-aea5-4ac1-a529-f5a508257288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:54.460083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa651da-aea5-4ac1-a529-f5a508257288 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:54.460367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa651da-aea5-4ac1-a529-f5a508257288 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:54.460641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa651da-aea5-4ac1-a529-f5a508257288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:54.471558 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa651da-aea5-4ac1-a529-f5a508257288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:54.473022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa651da-aea5-4ac1-a529-f5a508257288 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-0aa651da-aea5-4ac1-a529-f5a508257288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:54.486802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:54.487874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:54.489089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:54.489089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:54.489089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:54.489089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 545/1094] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:54.493634 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa651da-aea5-4ac1-a529-f5a508257288 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.528235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc2465-354a-48d0-8fd1-83fc9a6ed0b6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:54.549942 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:54.561835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffbc83-9e84-4edd-b945-2d810ad66428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:54.562437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffbc83-9e84-4edd-b945-2d810ad66428 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:54.562718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffbc83-9e84-4edd-b945-2d810ad66428 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:54.562993 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffbc83-9e84-4edd-b945-2d810ad66428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:54.592528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa651da-aea5-4ac1-a529-f5a508257288 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:54.626888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee8c99-1da2-40ff-b8d8-4fdc20667d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:54.627489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee8c99-1da2-40ff-b8d8-4fdc20667d59 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:54.627766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee8c99-1da2-40ff-b8d8-4fdc20667d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:54.628095 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee8c99-1da2-40ff-b8d8-4fdc20667d59 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:54.628699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee8c99-1da2-40ff-b8d8-4fdc20667d59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:54.629003 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee8c99-1da2-40ff-b8d8-4fdc20667d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:54.630449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee8c99-1da2-40ff-b8d8-4fdc20667d59 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:54.634318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffbc83-9e84-4edd-b945-2d810ad66428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:54.635807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffbc83-9e84-4edd-b945-2d810ad66428 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:54.643845 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 546/1095] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:54.649734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee8c99-1da2-40ff-b8d8-4fdc20667d59 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.691058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:54.736964 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 550/1096] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 22:20:54.743282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffbc83-9e84-4edd-b945-2d810ad66428 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.749548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee8c99-1da2-40ff-b8d8-4fdc20667d59 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:54.766867 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:54.783260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ee622-3f45-4a05-86ac-21a304836842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:54.783925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ee622-3f45-4a05-86ac-21a304836842 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:54.784223 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ee622-3f45-4a05-86ac-21a304836842 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:54.784497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ee622-3f45-4a05-86ac-21a304836842 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:54.791545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ee622-3f45-4a05-86ac-21a304836842 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:54.793018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ee622-3f45-4a05-86ac-21a304836842 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f83ee622-3f45-4a05-86ac-21a304836842 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:20:54.804272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:54.805963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:54.807126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:54.807126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:54.807126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:20:54.807126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:20:54.807126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:20:54.807126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:20:54.807126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:20:54.807126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:20:54.807126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 22:20:54.807126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:20:54.807126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 547/1097] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:20:54.811783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ee622-3f45-4a05-86ac-21a304836842 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.844228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffbc83-9e84-4edd-b945-2d810ad66428 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:54.888447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b380d-7c7a-4c0e-b031-a0be20271e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:54.889040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b380d-7c7a-4c0e-b031-a0be20271e14 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:54.889272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b380d-7c7a-4c0e-b031-a0be20271e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:54.889554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b380d-7c7a-4c0e-b031-a0be20271e14 None admin] PATH_INFO: `/v3/projects/029a0727429443f491e126247ff12b68` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:54.890690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b380d-7c7a-4c0e-b031-a0be20271e14 None admin] RBAC: Authorizing `identity:get_project(project_id=029a0727429443f491e126247ff12b68)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:54.892161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b380d-7c7a-4c0e-b031-a0be20271e14 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:54.893797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 551/1098] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/projects/029a0727429443f491e126247ff12b68 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:54.900052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b380d-7c7a-4c0e-b031-a0be20271e14 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.921597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ee622-3f45-4a05-86ac-21a304836842 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:54.923137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:54.956165 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65c4a3-2535-4a1f-8554-6803cb6ce9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:54.956760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65c4a3-2535-4a1f-8554-6803cb6ce9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:54.957060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65c4a3-2535-4a1f-8554-6803cb6ce9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:54.957332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65c4a3-2535-4a1f-8554-6803cb6ce9d5 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:54.958301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65c4a3-2535-4a1f-8554-6803cb6ce9d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:54.958415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65c4a3-2535-4a1f-8554-6803cb6ce9d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:54.959722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65c4a3-2535-4a1f-8554-6803cb6ce9d5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:54.976340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 548/1099] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:54.982696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65c4a3-2535-4a1f-8554-6803cb6ce9d5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:54.992158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:55.002757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b380d-7c7a-4c0e-b031-a0be20271e14 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:55.038153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c22fb-39b6-4d54-b89b-280d5406fdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:55.038748 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c22fb-39b6-4d54-b89b-280d5406fdef None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:55.039023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c22fb-39b6-4d54-b89b-280d5406fdef None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:55.039292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c22fb-39b6-4d54-b89b-280d5406fdef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:55.040582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c22fb-39b6-4d54-b89b-280d5406fdef None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=029a0727429443f491e126247ff12b68, user.id=46259818b9714181b24407fc6b9f0e07) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:55.040880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c22fb-39b6-4d54-b89b-280d5406fdef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:55.042292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c22fb-39b6-4d54-b89b-280d5406fdef None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:55.047865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 552/1100] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/role_assignments?user.id=46259818b9714181b24407fc6b9f0e07&scope.project.id=029a0727429443f491e126247ff12b68&role.id=108664305d2546949f7327b65d4cb667 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:20:55.083732 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65c4a3-2535-4a1f-8554-6803cb6ce9d5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:55.121324 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8605470-0a4e-4105-a33e-ce300a39c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:55.121947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8605470-0a4e-4105-a33e-ce300a39c515 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:55.122225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8605470-0a4e-4105-a33e-ce300a39c515 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:55.122498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8605470-0a4e-4105-a33e-ce300a39c515 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:55.152234 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:55.193490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8605470-0a4e-4105-a33e-ce300a39c515 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:55.194949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8605470-0a4e-4105-a33e-ce300a39c515 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:55.293429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 549/1101] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:20:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 22:20:55.299389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c22fb-39b6-4d54-b89b-280d5406fdef None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:55.397105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c22fb-39b6-4d54-b89b-280d5406fdef None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:55.431442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7c21-f649-402e-9e73-29e5ec4d2d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:55.431981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7c21-f649-402e-9e73-29e5ec4d2d5d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:55.432454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7c21-f649-402e-9e73-29e5ec4d2d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:55.432735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7c21-f649-402e-9e73-29e5ec4d2d5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:55.443213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b7c21-f649-402e-9e73-29e5ec4d2d5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:55.444697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b7c21-f649-402e-9e73-29e5ec4d2d5d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-bb8b7c21-f649-402e-9e73-29e5ec4d2d5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:20:55.458863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:55.459832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:20:55.461083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:20:55.461083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:20:55.461083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:55.461083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 553/1102] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:55.466298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8605470-0a4e-4105-a33e-ce300a39c515 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:55.525666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:55.565756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8605470-0a4e-4105-a33e-ce300a39c515 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:55.599652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d824dd-80ca-4306-ad57-60d5589d104b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:55.600277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d824dd-80ca-4306-ad57-60d5589d104b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:55.600558 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d824dd-80ca-4306-ad57-60d5589d104b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:55.600829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d824dd-80ca-4306-ad57-60d5589d104b None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:55.601463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d824dd-80ca-4306-ad57-60d5589d104b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:55.601766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d824dd-80ca-4306-ad57-60d5589d104b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:55.603909 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d824dd-80ca-4306-ad57-60d5589d104b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:55.615561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 550/1103] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:55.621465 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7c21-f649-402e-9e73-29e5ec4d2d5d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:55.631715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:55.729481 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7c21-f649-402e-9e73-29e5ec4d2d5d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:55.762183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edc690-2a2e-46bf-b1ae-d0310a706686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:55.762814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edc690-2a2e-46bf-b1ae-d0310a706686 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:55.763092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edc690-2a2e-46bf-b1ae-d0310a706686 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:55.763363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edc690-2a2e-46bf-b1ae-d0310a706686 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:55.770014 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edc690-2a2e-46bf-b1ae-d0310a706686 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:55.771453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edc690-2a2e-46bf-b1ae-d0310a706686 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f1edc690-2a2e-46bf-b1ae-d0310a706686 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:20:55.782526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:20:55.783438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:20:55.784647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:20:55.784647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:20:55.784647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:20:55.784647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:20:55.784647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:20:55.784647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:20:55.784647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:20:55.784647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:20:55.784647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 554/1104] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:20:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:20:55.789397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d824dd-80ca-4306-ad57-60d5589d104b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:55.888374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d824dd-80ca-4306-ad57-60d5589d104b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:55.907820 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:55.921128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adce6a-8a39-4d17-b529-8dbf58cda251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:55.921722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adce6a-8a39-4d17-b529-8dbf58cda251 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:55.922297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adce6a-8a39-4d17-b529-8dbf58cda251 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:55.922297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adce6a-8a39-4d17-b529-8dbf58cda251 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:55.922918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adce6a-8a39-4d17-b529-8dbf58cda251 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:55.923219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adce6a-8a39-4d17-b529-8dbf58cda251 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:55.924653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adce6a-8a39-4d17-b529-8dbf58cda251 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:55.937876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 551/1105] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:55.941737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edc690-2a2e-46bf-b1ae-d0310a706686 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:56.007944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:56.042143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edc690-2a2e-46bf-b1ae-d0310a706686 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:56.075958 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf57dac-9f61-4d0b-97f7-63eeb3196f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:56.076586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf57dac-9f61-4d0b-97f7-63eeb3196f88 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:56.076865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf57dac-9f61-4d0b-97f7-63eeb3196f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:56.077140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf57dac-9f61-4d0b-97f7-63eeb3196f88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:56.077856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf57dac-9f61-4d0b-97f7-63eeb3196f88 None admin] RBAC: Adding query filter params (role.id=6e6a34bbb7684b249265fcd10027d213, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=8effbd0670ef41f59b46feb9f0997554) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:20:56.078165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf57dac-9f61-4d0b-97f7-63eeb3196f88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:56.079581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf57dac-9f61-4d0b-97f7-63eeb3196f88 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:56.085182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 555/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:20:55 2022] GET /identity/v3/role_assignments?user.id=8effbd0670ef41f59b46feb9f0997554&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=6e6a34bbb7684b249265fcd10027d213 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:57.134855 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849a9ac-9f90-4f5d-8302-a8a1e963beb8 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:57.135424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849a9ac-9f90-4f5d-8302-a8a1e963beb8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:57.135424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849a9ac-9f90-4f5d-8302-a8a1e963beb8 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:57.136789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 552/1107] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:57.141895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99af23-60c3-4a5c-a105-5f27662dfae6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:57.142318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99af23-60c3-4a5c-a105-5f27662dfae6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:57.142451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99af23-60c3-4a5c-a105-5f27662dfae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:57.160213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ee99af23-60c3-4a5c-a105-5f27662dfae6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:57.161392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ee99af23-60c3-4a5c-a105-5f27662dfae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:57.225115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 556/1108] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:57.250186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849a9ac-9f90-4f5d-8302-a8a1e963beb8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:57.347379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849a9ac-9f90-4f5d-8302-a8a1e963beb8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:57.380646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a2a83-927d-4986-9056-4be7a82618ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:57.381247 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a2a83-927d-4986-9056-4be7a82618ab None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:57.381524 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a2a83-927d-4986-9056-4be7a82618ab None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:57.381796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a2a83-927d-4986-9056-4be7a82618ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:57.382893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a2a83-927d-4986-9056-4be7a82618ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:57.384463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a2a83-927d-4986-9056-4be7a82618ab None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:57.385981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 553/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:20:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:57.391531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99af23-60c3-4a5c-a105-5f27662dfae6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:57.448706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:57.492144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99af23-60c3-4a5c-a105-5f27662dfae6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:57.525403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e826da-754f-4c61-b920-473bb84be513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:57.526000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e826da-754f-4c61-b920-473bb84be513 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:57.526278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e826da-754f-4c61-b920-473bb84be513 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:57.526548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e826da-754f-4c61-b920-473bb84be513 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:57.527181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e826da-754f-4c61-b920-473bb84be513 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:57.528639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e826da-754f-4c61-b920-473bb84be513 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:57.656259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 557/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:20:57 2022] POST /identity/v3/users => generated 286 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:20:59.688836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa277a-e171-4102-97c0-336c0bf83f1b None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:59.689365 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa277a-e171-4102-97c0-336c0bf83f1b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:59.689365 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa277a-e171-4102-97c0-336c0bf83f1b None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:59.690702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 554/1111] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:20:59.695939 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dff094-eff0-493e-afd0-dbe057b9341b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:59.696507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dff094-eff0-493e-afd0-dbe057b9341b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:59.696874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dff094-eff0-493e-afd0-dbe057b9341b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:59.714931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c3dff094-eff0-493e-afd0-dbe057b9341b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:20:59.716146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c3dff094-eff0-493e-afd0-dbe057b9341b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:20:59.780796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 558/1112] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:20:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:20:59.803483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa277a-e171-4102-97c0-336c0bf83f1b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:59.853974 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:20:59.903640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa277a-e171-4102-97c0-336c0bf83f1b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:20:59.935379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc3844-3f17-4b1b-8be3-7edb55bdca7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:20:59.935988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc3844-3f17-4b1b-8be3-7edb55bdca7f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:20:59.936290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc3844-3f17-4b1b-8be3-7edb55bdca7f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:20:59.936562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc3844-3f17-4b1b-8be3-7edb55bdca7f None admin] PATH_INFO: `/v3/roles/b69a8899fce14a0d839745bb76b0f441` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:20:59.937957 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc3844-3f17-4b1b-8be3-7edb55bdca7f None admin] RBAC: Authorizing `identity:get_role(role_id=b69a8899fce14a0d839745bb76b0f441)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:20:59.939362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc3844-3f17-4b1b-8be3-7edb55bdca7f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:20:59.940643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 555/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:20:59 2022] GET /identity/v3/roles/b69a8899fce14a0d839745bb76b0f441 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:20:59.946981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dff094-eff0-493e-afd0-dbe057b9341b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:20:59.992360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:00.044398 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dff094-eff0-493e-afd0-dbe057b9341b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:00.077703 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13bd3d-5c08-4eff-873b-f0c6ba3fc7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:00.078317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13bd3d-5c08-4eff-873b-f0c6ba3fc7ac None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:00.078591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13bd3d-5c08-4eff-873b-f0c6ba3fc7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:00.078861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13bd3d-5c08-4eff-873b-f0c6ba3fc7ac None admin] PATH_INFO: `/v3/users/aa86131e9db746a191018e6fa29c3951` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:00.091429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13bd3d-5c08-4eff-873b-f0c6ba3fc7ac None admin] RBAC: Authorizing `identity:get_user(user_id=aa86131e9db746a191018e6fa29c3951)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:00.092893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13bd3d-5c08-4eff-873b-f0c6ba3fc7ac None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:00.094457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 559/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:20:59 2022] GET /identity/v3/users/aa86131e9db746a191018e6fa29c3951 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:00.100227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc3844-3f17-4b1b-8be3-7edb55bdca7f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:00.197355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc3844-3f17-4b1b-8be3-7edb55bdca7f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:00.212718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:00.230097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8bf61-c32f-4a8c-b8ac-479999df3576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:00.230683 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8bf61-c32f-4a8c-b8ac-479999df3576 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:00.230958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8bf61-c32f-4a8c-b8ac-479999df3576 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:00.231225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8bf61-c32f-4a8c-b8ac-479999df3576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:00.295224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8bf61-c32f-4a8c-b8ac-479999df3576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:00.296682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8bf61-c32f-4a8c-b8ac-479999df3576 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:00.389992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 556/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:21:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:21:00.396072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13bd3d-5c08-4eff-873b-f0c6ba3fc7ac None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:00.494983 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13bd3d-5c08-4eff-873b-f0c6ba3fc7ac None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:00.497455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:00.528438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d38a8-e610-4c52-91a5-77239418e299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:00.529206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d38a8-e610-4c52-91a5-77239418e299 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:00.529488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d38a8-e610-4c52-91a5-77239418e299 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:00.529849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d38a8-e610-4c52-91a5-77239418e299 None admin] PATH_INFO: `/v3/projects/029a0727429443f491e126247ff12b68` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:00.530977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d38a8-e610-4c52-91a5-77239418e299 None admin] RBAC: Authorizing `identity:get_project(project_id=029a0727429443f491e126247ff12b68)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:00.532447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d38a8-e610-4c52-91a5-77239418e299 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:00.534064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 560/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:21:00 2022] GET /identity/v3/projects/029a0727429443f491e126247ff12b68 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:00.541262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8bf61-c32f-4a8c-b8ac-479999df3576 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:00.558428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:00.637052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8bf61-c32f-4a8c-b8ac-479999df3576 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:00.671423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406c566-a6a1-4690-b12d-a5975df33f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:00.672079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406c566-a6a1-4690-b12d-a5975df33f48 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:00.672358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406c566-a6a1-4690-b12d-a5975df33f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:00.672627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406c566-a6a1-4690-b12d-a5975df33f48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:00.673347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406c566-a6a1-4690-b12d-a5975df33f48 None admin] RBAC: Adding query filter params (role.id=b69a8899fce14a0d839745bb76b0f441, scope.project.id=029a0727429443f491e126247ff12b68, user.id=aa86131e9db746a191018e6fa29c3951) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:00.673656 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406c566-a6a1-4690-b12d-a5975df33f48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:00.675067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406c566-a6a1-4690-b12d-a5975df33f48 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:00.680338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 557/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:21:00 2022] GET /identity/v3/role_assignments?user.id=aa86131e9db746a191018e6fa29c3951&scope.project.id=029a0727429443f491e126247ff12b68&role.id=b69a8899fce14a0d839745bb76b0f441 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:02.636442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa205-c7ad-48bd-ab7b-f324add00fe0 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:02.636977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa205-c7ad-48bd-ab7b-f324add00fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:02.636977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa205-c7ad-48bd-ab7b-f324add00fe0 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:02.638324 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 561/1118] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:02.643523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3d291-1ce0-47b4-b00b-d2fad8706f72 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:02.643931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3d291-1ce0-47b4-b00b-d2fad8706f72 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:02.644098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3d291-1ce0-47b4-b00b-d2fad8706f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:02.662057 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-08b3d291-1ce0-47b4-b00b-d2fad8706f72 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:02.663230 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-08b3d291-1ce0-47b4-b00b-d2fad8706f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:02.727841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 558/1119] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:02.750027 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fa205-c7ad-48bd-ab7b-f324add00fe0 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:02.848522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fa205-c7ad-48bd-ab7b-f324add00fe0 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:02.881856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787996cf-a382-4089-be2c-97e3c98bc2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:02.882474 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787996cf-a382-4089-be2c-97e3c98bc2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:02.882742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787996cf-a382-4089-be2c-97e3c98bc2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:02.883590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787996cf-a382-4089-be2c-97e3c98bc2d3 None admin] PATH_INFO: `/v3/roles/b69a8899fce14a0d839745bb76b0f441` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:02.885150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787996cf-a382-4089-be2c-97e3c98bc2d3 None admin] RBAC: Authorizing `identity:get_role(role_id=b69a8899fce14a0d839745bb76b0f441)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:02.886565 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787996cf-a382-4089-be2c-97e3c98bc2d3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:02.887806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 562/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:21:02 2022] GET /identity/v3/roles/b69a8899fce14a0d839745bb76b0f441 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:02.893508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3d291-1ce0-47b4-b00b-d2fad8706f72 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:02.904948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:02.989741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3d291-1ce0-47b4-b00b-d2fad8706f72 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:03.027232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36214f-98d8-4f8c-b1be-621256bd5525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:03.027861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36214f-98d8-4f8c-b1be-621256bd5525 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:03.028187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36214f-98d8-4f8c-b1be-621256bd5525 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:03.028463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36214f-98d8-4f8c-b1be-621256bd5525 None admin] PATH_INFO: `/v3/users/aa86131e9db746a191018e6fa29c3951` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:03.029579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36214f-98d8-4f8c-b1be-621256bd5525 None admin] RBAC: Authorizing `identity:get_user(user_id=aa86131e9db746a191018e6fa29c3951)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:03.031002 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36214f-98d8-4f8c-b1be-621256bd5525 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:03.032644 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 559/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:21:02 2022] GET /identity/v3/users/aa86131e9db746a191018e6fa29c3951 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:03.038462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787996cf-a382-4089-be2c-97e3c98bc2d3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:03.079869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:03.137512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787996cf-a382-4089-be2c-97e3c98bc2d3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:03.169742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73fbbe-ab2c-4348-a75c-1fbd5c7fa8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:03.170356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73fbbe-ab2c-4348-a75c-1fbd5c7fa8af None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:03.170633 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73fbbe-ab2c-4348-a75c-1fbd5c7fa8af None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:03.170904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73fbbe-ab2c-4348-a75c-1fbd5c7fa8af None admin] PATH_INFO: `/v3/projects/029a0727429443f491e126247ff12b68` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:03.172000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73fbbe-ab2c-4348-a75c-1fbd5c7fa8af None admin] RBAC: Authorizing `identity:get_project(project_id=029a0727429443f491e126247ff12b68)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:03.173441 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73fbbe-ab2c-4348-a75c-1fbd5c7fa8af None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:03.175030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 563/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:21:03 2022] GET /identity/v3/projects/029a0727429443f491e126247ff12b68 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:03.181539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36214f-98d8-4f8c-b1be-621256bd5525 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:03.257397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:03.277513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36214f-98d8-4f8c-b1be-621256bd5525 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:03.310237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016605f-d59e-4699-bb3e-d0bc9b127715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:03.310876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016605f-d59e-4699-bb3e-d0bc9b127715 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:03.311155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016605f-d59e-4699-bb3e-d0bc9b127715 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:03.311428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016605f-d59e-4699-bb3e-d0bc9b127715 None admin] PATH_INFO: `/v3/projects/029a0727429443f491e126247ff12b68/users/aa86131e9db746a191018e6fa29c3951/roles/b69a8899fce14a0d839745bb76b0f441` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:03.313560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016605f-d59e-4699-bb3e-d0bc9b127715 None admin] RBAC: Authorizing `identity:create_grant(project_id=029a0727429443f491e126247ff12b68, user_id=aa86131e9db746a191018e6fa29c3951, role_id=b69a8899fce14a0d839745bb76b0f441)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:03.314982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016605f-d59e-4699-bb3e-d0bc9b127715 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:03.363977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 560/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:21:03 2022] PUT /identity/v3/projects/029a0727429443f491e126247ff12b68/users/aa86131e9db746a191018e6fa29c3951/roles/b69a8899fce14a0d839745bb76b0f441 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:21:05.329516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68c5eb-ad19-4670-8776-43ed31ab5476 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:05.330061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68c5eb-ad19-4670-8776-43ed31ab5476 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:05.330061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68c5eb-ad19-4670-8776-43ed31ab5476 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:05.331423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 564/1124] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:05.337059 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eccf7e-324c-42f4-b9c4-65009f5c5d1c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:05.337433 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eccf7e-324c-42f4-b9c4-65009f5c5d1c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:05.337604 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eccf7e-324c-42f4-b9c4-65009f5c5d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:05.356361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-08eccf7e-324c-42f4-b9c4-65009f5c5d1c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:05.357533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-08eccf7e-324c-42f4-b9c4-65009f5c5d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:05.421190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 561/1125] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:05.443946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68c5eb-ad19-4670-8776-43ed31ab5476 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:05.544919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68c5eb-ad19-4670-8776-43ed31ab5476 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:05.553735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:05.578084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7578d6f-9242-45b3-9fbe-fc569ae9232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:05.578703 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578d6f-9242-45b3-9fbe-fc569ae9232a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:05.578980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578d6f-9242-45b3-9fbe-fc569ae9232a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:05.579295 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578d6f-9242-45b3-9fbe-fc569ae9232a None admin] PATH_INFO: `/v3/roles/b69a8899fce14a0d839745bb76b0f441` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:05.580843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7578d6f-9242-45b3-9fbe-fc569ae9232a None admin] RBAC: Authorizing `identity:get_role(role_id=b69a8899fce14a0d839745bb76b0f441)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:05.582154 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7578d6f-9242-45b3-9fbe-fc569ae9232a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:05.583504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 565/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:21:05 2022] GET /identity/v3/roles/b69a8899fce14a0d839745bb76b0f441 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:05.589648 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eccf7e-324c-42f4-b9c4-65009f5c5d1c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:05.654077 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:05.687418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eccf7e-324c-42f4-b9c4-65009f5c5d1c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:05.719695 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93c766-84f2-4005-82da-ec6e0fb8831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:05.720335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93c766-84f2-4005-82da-ec6e0fb8831c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:05.720612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93c766-84f2-4005-82da-ec6e0fb8831c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:05.720882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93c766-84f2-4005-82da-ec6e0fb8831c None admin] PATH_INFO: `/v3/users/aa86131e9db746a191018e6fa29c3951` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:05.721985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93c766-84f2-4005-82da-ec6e0fb8831c None admin] RBAC: Authorizing `identity:get_user(user_id=aa86131e9db746a191018e6fa29c3951)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:05.723386 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93c766-84f2-4005-82da-ec6e0fb8831c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:05.724983 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 562/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:21:05 2022] GET /identity/v3/users/aa86131e9db746a191018e6fa29c3951 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:05.731553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7578d6f-9242-45b3-9fbe-fc569ae9232a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:05.829659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7578d6f-9242-45b3-9fbe-fc569ae9232a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:05.864275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe852f5-1a0a-460b-af6e-ec17e3a303b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:05.864805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe852f5-1a0a-460b-af6e-ec17e3a303b3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:05.865082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe852f5-1a0a-460b-af6e-ec17e3a303b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:05.865352 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe852f5-1a0a-460b-af6e-ec17e3a303b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:05.909123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:05.931401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe852f5-1a0a-460b-af6e-ec17e3a303b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:05.932875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe852f5-1a0a-460b-af6e-ec17e3a303b3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:06.029054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 566/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:21:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:21:06.034973 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93c766-84f2-4005-82da-ec6e0fb8831c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:06.129916 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93c766-84f2-4005-82da-ec6e0fb8831c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:06.161433 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb6424-c45e-4e77-8565-12f3df04bf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:06.162033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb6424-c45e-4e77-8565-12f3df04bf31 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:06.162309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb6424-c45e-4e77-8565-12f3df04bf31 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:06.162579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb6424-c45e-4e77-8565-12f3df04bf31 None admin] PATH_INFO: `/v3/projects/029a0727429443f491e126247ff12b68` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:06.163668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb6424-c45e-4e77-8565-12f3df04bf31 None admin] RBAC: Authorizing `identity:get_project(project_id=029a0727429443f491e126247ff12b68)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:06.165672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb6424-c45e-4e77-8565-12f3df04bf31 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:06.167267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 563/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:21:06 2022] GET /identity/v3/projects/029a0727429443f491e126247ff12b68 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:06.173305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe852f5-1a0a-460b-af6e-ec17e3a303b3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:06.247624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:06.269518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe852f5-1a0a-460b-af6e-ec17e3a303b3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:06.302552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eddc7e-718a-439a-acb0-bfd63b71ee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:06.303143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eddc7e-718a-439a-acb0-bfd63b71ee3e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:06.303420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eddc7e-718a-439a-acb0-bfd63b71ee3e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:06.303692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eddc7e-718a-439a-acb0-bfd63b71ee3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:06.304431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eddc7e-718a-439a-acb0-bfd63b71ee3e None admin] RBAC: Adding query filter params (role.id=b69a8899fce14a0d839745bb76b0f441, scope.project.id=029a0727429443f491e126247ff12b68, user.id=aa86131e9db746a191018e6fa29c3951) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:06.304739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eddc7e-718a-439a-acb0-bfd63b71ee3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:06.306318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eddc7e-718a-439a-acb0-bfd63b71ee3e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:06.311850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 567/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:21:06 2022] GET /identity/v3/role_assignments?user.id=aa86131e9db746a191018e6fa29c3951&scope.project.id=029a0727429443f491e126247ff12b68&role.id=b69a8899fce14a0d839745bb76b0f441 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:08.312267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc2562-28d6-4d3e-afc6-5ad107440da2 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:08.312856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc2562-28d6-4d3e-afc6-5ad107440da2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:08.312856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc2562-28d6-4d3e-afc6-5ad107440da2 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:08.314137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 564/1131] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:08.319312 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec61e63-93d1-4905-8fec-6b17829a9011 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:08.319720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec61e63-93d1-4905-8fec-6b17829a9011 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:08.319860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec61e63-93d1-4905-8fec-6b17829a9011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:08.338037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0ec61e63-93d1-4905-8fec-6b17829a9011 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:08.339221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0ec61e63-93d1-4905-8fec-6b17829a9011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:08.403196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 568/1132] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:08.426229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc2562-28d6-4d3e-afc6-5ad107440da2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:08.430671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:08.525541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc2562-28d6-4d3e-afc6-5ad107440da2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:08.558304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552f1fc-aac8-410b-8e53-6cda580a3191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:08.558913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552f1fc-aac8-410b-8e53-6cda580a3191 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:08.559191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552f1fc-aac8-410b-8e53-6cda580a3191 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:08.559462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552f1fc-aac8-410b-8e53-6cda580a3191 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:08.560576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552f1fc-aac8-410b-8e53-6cda580a3191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:08.561978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552f1fc-aac8-410b-8e53-6cda580a3191 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:08.563538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 565/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:21:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:08.569195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec61e63-93d1-4905-8fec-6b17829a9011 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:08.635552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:08.665257 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec61e63-93d1-4905-8fec-6b17829a9011 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:08.697443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62e1ba-f935-4543-bb23-dded4932c1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:08.698041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62e1ba-f935-4543-bb23-dded4932c1fd None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:08.698306 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62e1ba-f935-4543-bb23-dded4932c1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:08.698576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62e1ba-f935-4543-bb23-dded4932c1fd None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:08.699224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62e1ba-f935-4543-bb23-dded4932c1fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:08.700681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62e1ba-f935-4543-bb23-dded4932c1fd None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:08.822265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 569/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:21:08 2022] POST /identity/v3/projects => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:21:10.845321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3f491-1e7a-407e-aff0-b894adffdc44 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:10.845888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3f491-1e7a-407e-aff0-b894adffdc44 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:10.845888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3f491-1e7a-407e-aff0-b894adffdc44 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:10.847189 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 566/1135] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:10.853006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ae792-00f1-4879-9ccb-033dce550f2d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:10.853394 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ae792-00f1-4879-9ccb-033dce550f2d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:10.853560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ae792-00f1-4879-9ccb-033dce550f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:10.874936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-822ae792-00f1-4879-9ccb-033dce550f2d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:10.877036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-822ae792-00f1-4879-9ccb-033dce550f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:10.944542 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 570/1136] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:10.968493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3f491-1e7a-407e-aff0-b894adffdc44 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:11.038875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:11.068857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3f491-1e7a-407e-aff0-b894adffdc44 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:11.104650 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273a784-e225-4ae5-aa45-fa4c41ea99e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:11.105190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273a784-e225-4ae5-aa45-fa4c41ea99e1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:11.105473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273a784-e225-4ae5-aa45-fa4c41ea99e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:11.105777 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273a784-e225-4ae5-aa45-fa4c41ea99e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:11.106887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273a784-e225-4ae5-aa45-fa4c41ea99e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:11.108412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273a784-e225-4ae5-aa45-fa4c41ea99e1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:11.109968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 567/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:21:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:11.115748 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ae792-00f1-4879-9ccb-033dce550f2d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:11.167350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:11.212917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ae792-00f1-4879-9ccb-033dce550f2d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:11.246858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5c2ab-abba-4f87-9bff-b75c8dd295cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:11.247456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5c2ab-abba-4f87-9bff-b75c8dd295cf None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:11.247736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5c2ab-abba-4f87-9bff-b75c8dd295cf None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:11.248033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5c2ab-abba-4f87-9bff-b75c8dd295cf None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:11.248688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5c2ab-abba-4f87-9bff-b75c8dd295cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:11.250129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5c2ab-abba-4f87-9bff-b75c8dd295cf None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:11.324978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 571/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:21:11 2022] POST /identity/v3/users => generated 286 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:21:13.360902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a5882-2279-4406-afc7-a1f86c2200c2 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:13.361429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a5882-2279-4406-afc7-a1f86c2200c2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:13.361429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a5882-2279-4406-afc7-a1f86c2200c2 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:13.362770 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 568/1139] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:13.368412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a6f67-a5d5-4e11-92ed-61c8efbd1666 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:13.368824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a6f67-a5d5-4e11-92ed-61c8efbd1666 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:13.368963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a6f67-a5d5-4e11-92ed-61c8efbd1666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:13.387229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-608a6f67-a5d5-4e11-92ed-61c8efbd1666 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:13.388442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-608a6f67-a5d5-4e11-92ed-61c8efbd1666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:13.451336 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 572/1140] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:13.474081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a5882-2279-4406-afc7-a1f86c2200c2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:13.571780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a5882-2279-4406-afc7-a1f86c2200c2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:13.605126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45526f89-b5d5-44d9-be7f-aaba5668b6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:13.605736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45526f89-b5d5-44d9-be7f-aaba5668b6ae None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:13.606033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45526f89-b5d5-44d9-be7f-aaba5668b6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:13.606307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45526f89-b5d5-44d9-be7f-aaba5668b6ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:13.613651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:13.616441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45526f89-b5d5-44d9-be7f-aaba5668b6ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-45526f89-b5d5-44d9-be7f-aaba5668b6ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:21:13.621371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:21:13.622441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:21:13.623333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 569/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:21:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:21:13.628459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a6f67-a5d5-4e11-92ed-61c8efbd1666 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:13.667783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:13.725814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a6f67-a5d5-4e11-92ed-61c8efbd1666 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:13.758390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347619d-113e-4477-af6f-4bef8bcf505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:13.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347619d-113e-4477-af6f-4bef8bcf505e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:13.759422 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347619d-113e-4477-af6f-4bef8bcf505e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:13.759694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347619d-113e-4477-af6f-4bef8bcf505e None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:13.760377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347619d-113e-4477-af6f-4bef8bcf505e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:13.760683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347619d-113e-4477-af6f-4bef8bcf505e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:13.762043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347619d-113e-4477-af6f-4bef8bcf505e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:13.769753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 573/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:21:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:13.776072 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45526f89-b5d5-44d9-be7f-aaba5668b6ae None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:13.872664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45526f89-b5d5-44d9-be7f-aaba5668b6ae None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:13.906163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d531341-ba42-44c1-9bf6-788b7af907df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:13.906774 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d531341-ba42-44c1-9bf6-788b7af907df None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:13.907053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d531341-ba42-44c1-9bf6-788b7af907df None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:13.907325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d531341-ba42-44c1-9bf6-788b7af907df None admin] PATH_INFO: `/v3/users/14e3bb17d2c64fe690b2d905ccccf59c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:13.920136 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d531341-ba42-44c1-9bf6-788b7af907df None admin] RBAC: Authorizing `identity:get_user(user_id=14e3bb17d2c64fe690b2d905ccccf59c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:13.921571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d531341-ba42-44c1-9bf6-788b7af907df None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:13.923176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 570/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:21:13 2022] GET /identity/v3/users/14e3bb17d2c64fe690b2d905ccccf59c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:13.928981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347619d-113e-4477-af6f-4bef8bcf505e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:14.024761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:14.027615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347619d-113e-4477-af6f-4bef8bcf505e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:14.060256 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f861513-ae88-42b6-94a1-7b026bf382ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:14.060847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f861513-ae88-42b6-94a1-7b026bf382ea None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:14.061121 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f861513-ae88-42b6-94a1-7b026bf382ea None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:14.061389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f861513-ae88-42b6-94a1-7b026bf382ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:14.126603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f861513-ae88-42b6-94a1-7b026bf382ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:14.127971 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f861513-ae88-42b6-94a1-7b026bf382ea None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:14.218951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:14.222386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 574/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:21:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:21:14.228408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d531341-ba42-44c1-9bf6-788b7af907df None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:14.285605 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:14.327531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d531341-ba42-44c1-9bf6-788b7af907df None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:14.358990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c5d49-144a-4acb-91e8-08e775c94e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:14.359595 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c5d49-144a-4acb-91e8-08e775c94e1a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:14.359871 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c5d49-144a-4acb-91e8-08e775c94e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:14.360173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c5d49-144a-4acb-91e8-08e775c94e1a None admin] PATH_INFO: `/v3/projects/057a4c3da8b745ff8a455bd0c75fb960` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:14.361270 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c5d49-144a-4acb-91e8-08e775c94e1a None admin] RBAC: Authorizing `identity:get_project(project_id=057a4c3da8b745ff8a455bd0c75fb960)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:14.362681 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c5d49-144a-4acb-91e8-08e775c94e1a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:14.364294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 571/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:21:14 2022] GET /identity/v3/projects/057a4c3da8b745ff8a455bd0c75fb960 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:14.370778 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f861513-ae88-42b6-94a1-7b026bf382ea None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:14.466205 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f861513-ae88-42b6-94a1-7b026bf382ea None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:14.499473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223af3a-4653-4fe0-8e34-28dbe826dcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:14.500118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223af3a-4653-4fe0-8e34-28dbe826dcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:14.500401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223af3a-4653-4fe0-8e34-28dbe826dcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:14.500679 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223af3a-4653-4fe0-8e34-28dbe826dcc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:14.501403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223af3a-4653-4fe0-8e34-28dbe826dcc5 None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=057a4c3da8b745ff8a455bd0c75fb960, user.id=14e3bb17d2c64fe690b2d905ccccf59c) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:14.501714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223af3a-4653-4fe0-8e34-28dbe826dcc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:14.503146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223af3a-4653-4fe0-8e34-28dbe826dcc5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:14.508537 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 575/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:21:14 2022] GET /identity/v3/role_assignments?user.id=14e3bb17d2c64fe690b2d905ccccf59c&scope.project.id=057a4c3da8b745ff8a455bd0c75fb960&role.id=108664305d2546949f7327b65d4cb667 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:16.453658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1310e36-08ec-4280-aa4f-e20a2a05a982 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:16.454188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1310e36-08ec-4280-aa4f-e20a2a05a982 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:16.454188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1310e36-08ec-4280-aa4f-e20a2a05a982 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:16.455530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 572/1147] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:16.460793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a1f25-3aa2-4c54-b352-967260f0c584 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:16.461232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a1f25-3aa2-4c54-b352-967260f0c584 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:16.461392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a1f25-3aa2-4c54-b352-967260f0c584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:16.478801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-098a1f25-3aa2-4c54-b352-967260f0c584 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:16.479994 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-098a1f25-3aa2-4c54-b352-967260f0c584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:16.536717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:16.544718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 576/1148] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:16.567169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1310e36-08ec-4280-aa4f-e20a2a05a982 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:16.666500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1310e36-08ec-4280-aa4f-e20a2a05a982 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:16.690848 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:16.699619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28885fc1-83c4-4ae1-ae02-a77620e169be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:16.700250 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28885fc1-83c4-4ae1-ae02-a77620e169be None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:16.700532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28885fc1-83c4-4ae1-ae02-a77620e169be None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:16.700806 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28885fc1-83c4-4ae1-ae02-a77620e169be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:16.710481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28885fc1-83c4-4ae1-ae02-a77620e169be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-28885fc1-83c4-4ae1-ae02-a77620e169be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:21:16.715253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:21:16.716360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:21:16.717212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 573/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:21:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:21:16.722288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a1f25-3aa2-4c54-b352-967260f0c584 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:16.817675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a1f25-3aa2-4c54-b352-967260f0c584 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:16.849719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1884b1e-61ad-4b53-a63e-ef55de64eea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:16.850311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1884b1e-61ad-4b53-a63e-ef55de64eea0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:16.850586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1884b1e-61ad-4b53-a63e-ef55de64eea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:16.850854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1884b1e-61ad-4b53-a63e-ef55de64eea0 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:16.852157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1884b1e-61ad-4b53-a63e-ef55de64eea0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:16.852463 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1884b1e-61ad-4b53-a63e-ef55de64eea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:16.853820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1884b1e-61ad-4b53-a63e-ef55de64eea0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:16.861510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 577/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:21:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:16.867367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28885fc1-83c4-4ae1-ae02-a77620e169be None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:16.965873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28885fc1-83c4-4ae1-ae02-a77620e169be None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:16.998709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b9c6b-4d95-47cc-ad9f-6e0fd8878d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:16.999325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9c6b-4d95-47cc-ad9f-6e0fd8878d5f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:16.999605 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9c6b-4d95-47cc-ad9f-6e0fd8878d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:16.999879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9c6b-4d95-47cc-ad9f-6e0fd8878d5f None admin] PATH_INFO: `/v3/users/14e3bb17d2c64fe690b2d905ccccf59c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:17.001020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b9c6b-4d95-47cc-ad9f-6e0fd8878d5f None admin] RBAC: Authorizing `identity:get_user(user_id=14e3bb17d2c64fe690b2d905ccccf59c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:17.002434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b9c6b-4d95-47cc-ad9f-6e0fd8878d5f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:17.003986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 574/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:21:16 2022] GET /identity/v3/users/14e3bb17d2c64fe690b2d905ccccf59c => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:17.010488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1884b1e-61ad-4b53-a63e-ef55de64eea0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:17.065910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:17.108735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1884b1e-61ad-4b53-a63e-ef55de64eea0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:17.141530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace1955-3736-453b-b339-8036dcf80a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:17.142152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace1955-3736-453b-b339-8036dcf80a3a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:17.142433 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace1955-3736-453b-b339-8036dcf80a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:17.142710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace1955-3736-453b-b339-8036dcf80a3a None admin] PATH_INFO: `/v3/projects/057a4c3da8b745ff8a455bd0c75fb960` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:17.143822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace1955-3736-453b-b339-8036dcf80a3a None admin] RBAC: Authorizing `identity:get_project(project_id=057a4c3da8b745ff8a455bd0c75fb960)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:17.145278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace1955-3736-453b-b339-8036dcf80a3a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:17.146962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 578/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:21:17 2022] GET /identity/v3/projects/057a4c3da8b745ff8a455bd0c75fb960 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:17.153241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b9c6b-4d95-47cc-ad9f-6e0fd8878d5f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:17.206619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:17.251028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b9c6b-4d95-47cc-ad9f-6e0fd8878d5f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:17.284249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6476e-3743-46d5-b11d-5a0afb7d10a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:17.284892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6476e-3743-46d5-b11d-5a0afb7d10a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:17.285174 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6476e-3743-46d5-b11d-5a0afb7d10a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:17.285447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6476e-3743-46d5-b11d-5a0afb7d10a1 None admin] PATH_INFO: `/v3/projects/057a4c3da8b745ff8a455bd0c75fb960/users/14e3bb17d2c64fe690b2d905ccccf59c/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:17.287339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6476e-3743-46d5-b11d-5a0afb7d10a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=057a4c3da8b745ff8a455bd0c75fb960, user_id=14e3bb17d2c64fe690b2d905ccccf59c, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:17.289350 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6476e-3743-46d5-b11d-5a0afb7d10a1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:17.381319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 575/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:21:17 2022] PUT /identity/v3/projects/057a4c3da8b745ff8a455bd0c75fb960/users/14e3bb17d2c64fe690b2d905ccccf59c/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:21:19.339687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189dc68-7a1a-41db-b675-5539cebfd5d4 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:19.340282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189dc68-7a1a-41db-b675-5539cebfd5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:19.340282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189dc68-7a1a-41db-b675-5539cebfd5d4 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:19.341588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 579/1154] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:19.346883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa291b1-d1b6-43ff-950c-9709cbcf54f4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:19.347310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa291b1-d1b6-43ff-950c-9709cbcf54f4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:19.347418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa291b1-d1b6-43ff-950c-9709cbcf54f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:19.364998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-faa291b1-d1b6-43ff-950c-9709cbcf54f4 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:19.366185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-faa291b1-d1b6-43ff-950c-9709cbcf54f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:19.435781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 576/1155] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:21:19.458371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189dc68-7a1a-41db-b675-5539cebfd5d4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:19.558149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189dc68-7a1a-41db-b675-5539cebfd5d4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:19.580409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:19.591064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e151e7-1400-4c65-bd52-9096b959e89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:19.591650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e151e7-1400-4c65-bd52-9096b959e89e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:19.591929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e151e7-1400-4c65-bd52-9096b959e89e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:19.597795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e151e7-1400-4c65-bd52-9096b959e89e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:19.602404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e151e7-1400-4c65-bd52-9096b959e89e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-64e151e7-1400-4c65-bd52-9096b959e89e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:21:19.607023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:21:19.607947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:21:19.609066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 580/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:21:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:21:19.614265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa291b1-d1b6-43ff-950c-9709cbcf54f4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:19.659564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:19.712562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa291b1-d1b6-43ff-950c-9709cbcf54f4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:19.745445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93172d-a81d-459b-93b3-da0c6524d3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:19.746034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93172d-a81d-459b-93b3-da0c6524d3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:19.746560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93172d-a81d-459b-93b3-da0c6524d3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:19.746829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93172d-a81d-459b-93b3-da0c6524d3b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:19.747478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93172d-a81d-459b-93b3-da0c6524d3b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:19.747778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93172d-a81d-459b-93b3-da0c6524d3b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:19.749156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93172d-a81d-459b-93b3-da0c6524d3b6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:19.756823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 577/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:21:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:21:19.763298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e151e7-1400-4c65-bd52-9096b959e89e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:19.859792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e151e7-1400-4c65-bd52-9096b959e89e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:19.892750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e5626-f732-49cf-a759-3bfab335a132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:19.893362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e5626-f732-49cf-a759-3bfab335a132 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:19.893638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e5626-f732-49cf-a759-3bfab335a132 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:19.893910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e5626-f732-49cf-a759-3bfab335a132 None admin] PATH_INFO: `/v3/users/14e3bb17d2c64fe690b2d905ccccf59c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:19.895012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e5626-f732-49cf-a759-3bfab335a132 None admin] RBAC: Authorizing `identity:get_user(user_id=14e3bb17d2c64fe690b2d905ccccf59c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:19.896476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e5626-f732-49cf-a759-3bfab335a132 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:19.898215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 581/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:21:19 2022] GET /identity/v3/users/14e3bb17d2c64fe690b2d905ccccf59c => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:19.904511 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93172d-a81d-459b-93b3-da0c6524d3b6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:20.000807 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93172d-a81d-459b-93b3-da0c6524d3b6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:20.009603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:20.036351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777a485-d7b6-4080-9179-bc8f9394e07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:20.036940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777a485-d7b6-4080-9179-bc8f9394e07d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:20.037248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777a485-d7b6-4080-9179-bc8f9394e07d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:20.037716 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777a485-d7b6-4080-9179-bc8f9394e07d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:20.102082 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777a485-d7b6-4080-9179-bc8f9394e07d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:20.103606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777a485-d7b6-4080-9179-bc8f9394e07d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:20.200873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 578/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:21:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:21:20.206686 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e5626-f732-49cf-a759-3bfab335a132 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:20.258054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:20.302128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e5626-f732-49cf-a759-3bfab335a132 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:20.334250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366d0c2-e491-4472-b8ab-8d2f157cf357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:20.334861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366d0c2-e491-4472-b8ab-8d2f157cf357 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:20.335134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366d0c2-e491-4472-b8ab-8d2f157cf357 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:20.335399 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366d0c2-e491-4472-b8ab-8d2f157cf357 None admin] PATH_INFO: `/v3/projects/057a4c3da8b745ff8a455bd0c75fb960` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:20.337038 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366d0c2-e491-4472-b8ab-8d2f157cf357 None admin] RBAC: Authorizing `identity:get_project(project_id=057a4c3da8b745ff8a455bd0c75fb960)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:20.338446 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366d0c2-e491-4472-b8ab-8d2f157cf357 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:20.340057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 582/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:21:20 2022] GET /identity/v3/projects/057a4c3da8b745ff8a455bd0c75fb960 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:20.346050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777a485-d7b6-4080-9179-bc8f9394e07d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:20.354830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:20.442676 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777a485-d7b6-4080-9179-bc8f9394e07d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:20.475409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edd218-0925-40a1-b252-51abe7f389ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:20.475944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd218-0925-40a1-b252-51abe7f389ef None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:20.476404 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd218-0925-40a1-b252-51abe7f389ef None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:20.476678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd218-0925-40a1-b252-51abe7f389ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:20.477403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edd218-0925-40a1-b252-51abe7f389ef None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=057a4c3da8b745ff8a455bd0c75fb960, user.id=14e3bb17d2c64fe690b2d905ccccf59c) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:20.477713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edd218-0925-40a1-b252-51abe7f389ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:20.479121 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edd218-0925-40a1-b252-51abe7f389ef None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:20.484711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 579/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:21:20 2022] GET /identity/v3/role_assignments?user.id=14e3bb17d2c64fe690b2d905ccccf59c&scope.project.id=057a4c3da8b745ff8a455bd0c75fb960&role.id=108664305d2546949f7327b65d4cb667 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:22.487808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60929271-9b51-43db-9b6b-44ab9b0e4dee None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:22.488390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60929271-9b51-43db-9b6b-44ab9b0e4dee None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:22.488390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60929271-9b51-43db-9b6b-44ab9b0e4dee None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:22.489783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 583/1162] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:22.494972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62e7be-774f-4bbd-af7b-303c04673738 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:22.495411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62e7be-774f-4bbd-af7b-303c04673738 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:22.495573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62e7be-774f-4bbd-af7b-303c04673738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:22.514149 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ed62e7be-774f-4bbd-af7b-303c04673738 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:22.515328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ed62e7be-774f-4bbd-af7b-303c04673738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:22.581122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:22.586139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 580/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:22.608947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60929271-9b51-43db-9b6b-44ab9b0e4dee None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:22.709700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60929271-9b51-43db-9b6b-44ab9b0e4dee None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:22.730014 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:22.744190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef93faa-8fc1-4d5a-9db3-711a9760d0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:22.744779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef93faa-8fc1-4d5a-9db3-711a9760d0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:22.745052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef93faa-8fc1-4d5a-9db3-711a9760d0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:22.745321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef93faa-8fc1-4d5a-9db3-711a9760d0f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:22.819615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef93faa-8fc1-4d5a-9db3-711a9760d0f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:22.821817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef93faa-8fc1-4d5a-9db3-711a9760d0f9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:22.916133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 584/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:21:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:21:22.922053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62e7be-774f-4bbd-af7b-303c04673738 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:23.016295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62e7be-774f-4bbd-af7b-303c04673738 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:23.049165 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb943e0-285d-4106-b75b-2f02942bdc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:23.049771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb943e0-285d-4106-b75b-2f02942bdc6f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:23.050044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb943e0-285d-4106-b75b-2f02942bdc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:23.050310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb943e0-285d-4106-b75b-2f02942bdc6f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:23.057340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb943e0-285d-4106-b75b-2f02942bdc6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:23.058760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb943e0-285d-4106-b75b-2f02942bdc6f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-aeb943e0-285d-4106-b75b-2f02942bdc6f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:21:23.069620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:21:23.070818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:21:23.072097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 581/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 22:21:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:21:23.076750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef93faa-8fc1-4d5a-9db3-711a9760d0f9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:23.097697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:23.172979 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef93faa-8fc1-4d5a-9db3-711a9760d0f9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:23.205205 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228935d-8c6e-43b9-a9c3-a4624784a03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:23.205798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228935d-8c6e-43b9-a9c3-a4624784a03d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:23.206074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228935d-8c6e-43b9-a9c3-a4624784a03d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:23.206344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228935d-8c6e-43b9-a9c3-a4624784a03d None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:23.206975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228935d-8c6e-43b9-a9c3-a4624784a03d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:23.207276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228935d-8c6e-43b9-a9c3-a4624784a03d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:23.208658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228935d-8c6e-43b9-a9c3-a4624784a03d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:23.216126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 585/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 22:21:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:25.204507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96e17c-00a0-40d2-bdd7-c683d6574e1a None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:25.205037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96e17c-00a0-40d2-bdd7-c683d6574e1a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:25.205037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96e17c-00a0-40d2-bdd7-c683d6574e1a None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:25.206303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 582/1167] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:25.211408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41b616-72f8-441f-9404-05d7d6d25f72 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:25.211829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41b616-72f8-441f-9404-05d7d6d25f72 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:25.211961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41b616-72f8-441f-9404-05d7d6d25f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:25.229975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8e41b616-72f8-441f-9404-05d7d6d25f72 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:25.231157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8e41b616-72f8-441f-9404-05d7d6d25f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:25.295861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 586/1168] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:25.318227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96e17c-00a0-40d2-bdd7-c683d6574e1a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:25.376214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:25.417037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96e17c-00a0-40d2-bdd7-c683d6574e1a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:25.449074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67e077-5994-4eb2-ba37-e3c8bd8283a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:25.449650 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67e077-5994-4eb2-ba37-e3c8bd8283a5 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:25.449925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67e077-5994-4eb2-ba37-e3c8bd8283a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:25.450191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67e077-5994-4eb2-ba37-e3c8bd8283a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:25.450829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67e077-5994-4eb2-ba37-e3c8bd8283a5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:25.452204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67e077-5994-4eb2-ba37-e3c8bd8283a5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:25.525362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 583/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Sun Aug 7 22:21:25 2022] POST /identity/v3/domains => generated 267 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:21:27.531775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd98832-5948-4ad5-a437-63590d6b1934 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:27.532428 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd98832-5948-4ad5-a437-63590d6b1934 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:27.532428 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd98832-5948-4ad5-a437-63590d6b1934 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:27.533728 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 587/1170] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:27.538764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf5f2b-a7e0-4b34-aa6d-e27d27eb153a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:27.539165 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf5f2b-a7e0-4b34-aa6d-e27d27eb153a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:27.539314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf5f2b-a7e0-4b34-aa6d-e27d27eb153a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:27.557613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a7cf5f2b-a7e0-4b34-aa6d-e27d27eb153a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:27.558783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a7cf5f2b-a7e0-4b34-aa6d-e27d27eb153a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:27.622376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 584/1171] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:27.645440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd98832-5948-4ad5-a437-63590d6b1934 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:27.661002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:27.745961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd98832-5948-4ad5-a437-63590d6b1934 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:27.779186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb4f29-9ec4-48c7-ab31-dd791e29e0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:27.779796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb4f29-9ec4-48c7-ab31-dd791e29e0fd None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:27.780110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb4f29-9ec4-48c7-ab31-dd791e29e0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:27.780390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb4f29-9ec4-48c7-ab31-dd791e29e0fd None admin] PATH_INFO: `/v3/domains/c1b61edfd4f4488a93b95edf6fc11aab` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:27.790529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb4f29-9ec4-48c7-ab31-dd791e29e0fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=c1b61edfd4f4488a93b95edf6fc11aab)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:27.791906 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb4f29-9ec4-48c7-ab31-dd791e29e0fd None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:27.793491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 588/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Sun Aug 7 22:21:27 2022] GET /identity/v3/domains/c1b61edfd4f4488a93b95edf6fc11aab => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:27.799332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf5f2b-a7e0-4b34-aa6d-e27d27eb153a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:27.848955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:27.898150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf5f2b-a7e0-4b34-aa6d-e27d27eb153a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:27.932155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b10e8b-29b9-4b04-ac33-b436031f6224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:27.932743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b10e8b-29b9-4b04-ac33-b436031f6224 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:27.934288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b10e8b-29b9-4b04-ac33-b436031f6224 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:27.934288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b10e8b-29b9-4b04-ac33-b436031f6224 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:27.934288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b10e8b-29b9-4b04-ac33-b436031f6224 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:27.935409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b10e8b-29b9-4b04-ac33-b436031f6224 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:28.022140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 585/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:21:27 2022] POST /identity/v3/projects => generated 361 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:21:30.047552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c9066-8303-4018-bdf6-ad3349f60178 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:30.048124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c9066-8303-4018-bdf6-ad3349f60178 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:30.048124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c9066-8303-4018-bdf6-ad3349f60178 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:30.049491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 589/1174] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:30.055323 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250d979-0fc7-4883-ac8b-e906670fce66 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:30.055771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250d979-0fc7-4883-ac8b-e906670fce66 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:30.055875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250d979-0fc7-4883-ac8b-e906670fce66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:30.074109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3250d979-0fc7-4883-ac8b-e906670fce66 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:30.075282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3250d979-0fc7-4883-ac8b-e906670fce66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:30.141472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 586/1175] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 22:21:30.166157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c9066-8303-4018-bdf6-ad3349f60178 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:30.240420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:30.266091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c9066-8303-4018-bdf6-ad3349f60178 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:30.299570 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ecdfad-7f9a-4cc9-925f-11326aa01b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:30.300213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecdfad-7f9a-4cc9-925f-11326aa01b8a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:30.300496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecdfad-7f9a-4cc9-925f-11326aa01b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:30.300771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecdfad-7f9a-4cc9-925f-11326aa01b8a None admin] PATH_INFO: `/v3/domains/c1b61edfd4f4488a93b95edf6fc11aab` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:30.301883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ecdfad-7f9a-4cc9-925f-11326aa01b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=c1b61edfd4f4488a93b95edf6fc11aab)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:30.303318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ecdfad-7f9a-4cc9-925f-11326aa01b8a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:30.305613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 590/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Sun Aug 7 22:21:30 2022] GET /identity/v3/domains/c1b61edfd4f4488a93b95edf6fc11aab => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:30.311517 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250d979-0fc7-4883-ac8b-e906670fce66 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:30.340928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:30.407947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250d979-0fc7-4883-ac8b-e906670fce66 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:30.441244 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c340e20-cb10-4bf0-8b8e-41fc5b10ea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:30.441838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c340e20-cb10-4bf0-8b8e-41fc5b10ea62 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:30.442119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c340e20-cb10-4bf0-8b8e-41fc5b10ea62 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:30.442394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c340e20-cb10-4bf0-8b8e-41fc5b10ea62 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:30.443031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c340e20-cb10-4bf0-8b8e-41fc5b10ea62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:30.444486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c340e20-cb10-4bf0-8b8e-41fc5b10ea62 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:30.516892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 587/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:21:30 2022] POST /identity/v3/users => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:21:32.546784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89026a39-2394-49d6-9140-b2b03de28f1e None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:32.547339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89026a39-2394-49d6-9140-b2b03de28f1e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:32.547339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89026a39-2394-49d6-9140-b2b03de28f1e None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:32.548717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 591/1178] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:32.553941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1c754-d644-4db2-92c4-7d5237d97399 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:32.554336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1c754-d644-4db2-92c4-7d5237d97399 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:32.554537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1c754-d644-4db2-92c4-7d5237d97399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:32.572739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-72a1c754-d644-4db2-92c4-7d5237d97399 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:32.574263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-72a1c754-d644-4db2-92c4-7d5237d97399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:32.639709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 588/1179] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:32.663310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89026a39-2394-49d6-9140-b2b03de28f1e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:32.774709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89026a39-2394-49d6-9140-b2b03de28f1e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:32.808916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e9ce7-1a3d-4055-85cd-325a2325d43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:32.809464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e9ce7-1a3d-4055-85cd-325a2325d43c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:32.809745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e9ce7-1a3d-4055-85cd-325a2325d43c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:32.810021 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e9ce7-1a3d-4055-85cd-325a2325d43c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:32.820769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e9ce7-1a3d-4055-85cd-325a2325d43c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-945e9ce7-1a3d-4055-85cd-325a2325d43c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:21:32.825625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:21:32.826714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:21:32.827747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 592/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:21:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:21:32.832926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1c754-d644-4db2-92c4-7d5237d97399 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:32.854435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:32.956636 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1c754-d644-4db2-92c4-7d5237d97399 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:32.992605 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d0592-3090-4a87-a360-754fd0ed3576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:32.993246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0592-3090-4a87-a360-754fd0ed3576 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:32.993530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0592-3090-4a87-a360-754fd0ed3576 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:32.993809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0592-3090-4a87-a360-754fd0ed3576 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:32.999090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d0592-3090-4a87-a360-754fd0ed3576 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:33.002386 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d0592-3090-4a87-a360-754fd0ed3576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:33.003794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d0592-3090-4a87-a360-754fd0ed3576 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:33.013190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 589/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:21:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:21:33.018973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e9ce7-1a3d-4055-85cd-325a2325d43c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:33.023171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:33.128181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e9ce7-1a3d-4055-85cd-325a2325d43c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:33.172969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b866649-9ad9-499f-8885-82354e3b2920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:33.173931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866649-9ad9-499f-8885-82354e3b2920 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:33.174385 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866649-9ad9-499f-8885-82354e3b2920 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:33.174830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866649-9ad9-499f-8885-82354e3b2920 None admin] PATH_INFO: `/v3/users/5384e6c51181428cab0c2e9aeca4b38c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:33.193929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b866649-9ad9-499f-8885-82354e3b2920 None admin] RBAC: Authorizing `identity:get_user(user_id=5384e6c51181428cab0c2e9aeca4b38c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:33.195506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b866649-9ad9-499f-8885-82354e3b2920 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:33.197487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 593/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:21:33 2022] GET /identity/v3/users/5384e6c51181428cab0c2e9aeca4b38c => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:33.203282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d0592-3090-4a87-a360-754fd0ed3576 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:33.291452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:33.311116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d0592-3090-4a87-a360-754fd0ed3576 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:33.348037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c860568-9623-4975-9afd-f524dd28abda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:33.348647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c860568-9623-4975-9afd-f524dd28abda None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:33.348927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c860568-9623-4975-9afd-f524dd28abda None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:33.349200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c860568-9623-4975-9afd-f524dd28abda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:33.434129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c860568-9623-4975-9afd-f524dd28abda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:33.435494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c860568-9623-4975-9afd-f524dd28abda None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:33.519316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:33.539580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 590/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:21:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:21:33.545459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b866649-9ad9-499f-8885-82354e3b2920 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:33.603007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:33.642445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b866649-9ad9-499f-8885-82354e3b2920 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:33.676127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32211ce9-11ed-436e-bd62-1da53df6b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:33.676809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32211ce9-11ed-436e-bd62-1da53df6b5ff None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:33.677094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32211ce9-11ed-436e-bd62-1da53df6b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:33.677369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32211ce9-11ed-436e-bd62-1da53df6b5ff None admin] PATH_INFO: `/v3/projects/34472727bdad4c0496400ed82ab60cb1` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:33.678492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32211ce9-11ed-436e-bd62-1da53df6b5ff None admin] RBAC: Authorizing `identity:get_project(project_id=34472727bdad4c0496400ed82ab60cb1)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:33.679995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32211ce9-11ed-436e-bd62-1da53df6b5ff None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:33.681647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 594/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:21:33 2022] GET /identity/v3/projects/34472727bdad4c0496400ed82ab60cb1 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:33.687733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c860568-9623-4975-9afd-f524dd28abda None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:33.790023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c860568-9623-4975-9afd-f524dd28abda None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:33.822672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38021b14-f8b2-4be1-88af-f6efc464456c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:33.823263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38021b14-f8b2-4be1-88af-f6efc464456c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:33.823543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38021b14-f8b2-4be1-88af-f6efc464456c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:33.823819 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38021b14-f8b2-4be1-88af-f6efc464456c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:33.824569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38021b14-f8b2-4be1-88af-f6efc464456c None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=34472727bdad4c0496400ed82ab60cb1, user.id=5384e6c51181428cab0c2e9aeca4b38c) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:33.824882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38021b14-f8b2-4be1-88af-f6efc464456c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:33.826297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38021b14-f8b2-4be1-88af-f6efc464456c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:33.831480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 591/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:21:33 2022] GET /identity/v3/role_assignments?user.id=5384e6c51181428cab0c2e9aeca4b38c&scope.project.id=34472727bdad4c0496400ed82ab60cb1&role.id=108664305d2546949f7327b65d4cb667 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:35.793813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93fb62-7a6b-402a-b845-483058726711 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:35.794389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93fb62-7a6b-402a-b845-483058726711 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:35.794389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93fb62-7a6b-402a-b845-483058726711 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:35.795700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 595/1186] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:35.800805 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac05b7-addf-4813-b35d-28320d920ede None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:35.801233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac05b7-addf-4813-b35d-28320d920ede None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:35.801366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac05b7-addf-4813-b35d-28320d920ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:35.820406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a6ac05b7-addf-4813-b35d-28320d920ede None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:35.821775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a6ac05b7-addf-4813-b35d-28320d920ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:35.860078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:35.887276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 592/1187] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:35.908663 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93fb62-7a6b-402a-b845-483058726711 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:36.008444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93fb62-7a6b-402a-b845-483058726711 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:36.033861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:36.041794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f209c-1bcd-446c-b41f-11e03a765df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:36.042408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f209c-1bcd-446c-b41f-11e03a765df0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:36.042683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f209c-1bcd-446c-b41f-11e03a765df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:36.045438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f209c-1bcd-446c-b41f-11e03a765df0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:36.052685 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f209c-1bcd-446c-b41f-11e03a765df0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-dc2f209c-1bcd-446c-b41f-11e03a765df0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:21:36.057603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:21:36.058562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:36.059411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:21:36.060100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 596/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:21:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:21:36.064862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac05b7-addf-4813-b35d-28320d920ede None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:36.164787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac05b7-addf-4813-b35d-28320d920ede None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:36.197632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac333a2-22ab-473c-abc1-40f12ee7998f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:36.198230 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac333a2-22ab-473c-abc1-40f12ee7998f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:36.198512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac333a2-22ab-473c-abc1-40f12ee7998f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:36.198787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac333a2-22ab-473c-abc1-40f12ee7998f None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:36.199444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac333a2-22ab-473c-abc1-40f12ee7998f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:36.199748 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac333a2-22ab-473c-abc1-40f12ee7998f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:36.201160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac333a2-22ab-473c-abc1-40f12ee7998f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:36.208866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 593/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:21:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:21:36.215268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f209c-1bcd-446c-b41f-11e03a765df0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:36.311653 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f209c-1bcd-446c-b41f-11e03a765df0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:36.344445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848e743-0f5c-40e3-b3c1-591832625034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:36.345045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848e743-0f5c-40e3-b3c1-591832625034 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:36.345321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848e743-0f5c-40e3-b3c1-591832625034 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:36.345591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848e743-0f5c-40e3-b3c1-591832625034 None admin] PATH_INFO: `/v3/users/5384e6c51181428cab0c2e9aeca4b38c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:36.346699 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848e743-0f5c-40e3-b3c1-591832625034 None admin] RBAC: Authorizing `identity:get_user(user_id=5384e6c51181428cab0c2e9aeca4b38c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:36.348143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848e743-0f5c-40e3-b3c1-591832625034 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:36.349683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 597/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:21:36 2022] GET /identity/v3/users/5384e6c51181428cab0c2e9aeca4b38c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:36.355400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac333a2-22ab-473c-abc1-40f12ee7998f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:36.390547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:36.451951 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac333a2-22ab-473c-abc1-40f12ee7998f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:36.484509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bcfca-4209-4d4b-a800-a3386248749e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:36.485118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bcfca-4209-4d4b-a800-a3386248749e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:36.485394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bcfca-4209-4d4b-a800-a3386248749e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:36.485666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bcfca-4209-4d4b-a800-a3386248749e None admin] PATH_INFO: `/v3/projects/34472727bdad4c0496400ed82ab60cb1` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:36.486769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bcfca-4209-4d4b-a800-a3386248749e None admin] RBAC: Authorizing `identity:get_project(project_id=34472727bdad4c0496400ed82ab60cb1)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:36.488213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bcfca-4209-4d4b-a800-a3386248749e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:36.489806 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 594/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:21:36 2022] GET /identity/v3/projects/34472727bdad4c0496400ed82ab60cb1 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:36.495647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848e743-0f5c-40e3-b3c1-591832625034 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:36.550985 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:36.593620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848e743-0f5c-40e3-b3c1-591832625034 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:36.626367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d7276-52f7-40ce-8d54-c65b8e6af6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:36.627019 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d7276-52f7-40ce-8d54-c65b8e6af6c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:36.627298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d7276-52f7-40ce-8d54-c65b8e6af6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:36.627572 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d7276-52f7-40ce-8d54-c65b8e6af6c7 None admin] PATH_INFO: `/v3/projects/34472727bdad4c0496400ed82ab60cb1/users/5384e6c51181428cab0c2e9aeca4b38c/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:36.629520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d7276-52f7-40ce-8d54-c65b8e6af6c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=34472727bdad4c0496400ed82ab60cb1, user_id=5384e6c51181428cab0c2e9aeca4b38c, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:36.630943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d7276-52f7-40ce-8d54-c65b8e6af6c7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:36.684090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 598/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:21:36 2022] PUT /identity/v3/projects/34472727bdad4c0496400ed82ab60cb1/users/5384e6c51181428cab0c2e9aeca4b38c/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:21:38.638727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41258d-c4bc-4ba0-a3c1-45201886f227 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:38.639319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41258d-c4bc-4ba0-a3c1-45201886f227 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:38.639319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41258d-c4bc-4ba0-a3c1-45201886f227 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:38.640626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 595/1193] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:21:38.645736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc06e56-80b8-43b1-b127-ca11d3c1c220 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:38.646191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc06e56-80b8-43b1-b127-ca11d3c1c220 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:38.646294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc06e56-80b8-43b1-b127-ca11d3c1c220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:38.664833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8cc06e56-80b8-43b1-b127-ca11d3c1c220 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:21:38.666020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8cc06e56-80b8-43b1-b127-ca11d3c1c220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:21:38.730918 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 599/1194] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:21:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:21:38.753854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41258d-c4bc-4ba0-a3c1-45201886f227 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:38.853470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41258d-c4bc-4ba0-a3c1-45201886f227 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:38.855737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:38.887001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49335d47-e0ec-4b70-ba5d-75899ed8ee4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:38.887619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49335d47-e0ec-4b70-ba5d-75899ed8ee4c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:38.887901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49335d47-e0ec-4b70-ba5d-75899ed8ee4c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:38.888207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49335d47-e0ec-4b70-ba5d-75899ed8ee4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:38.898783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49335d47-e0ec-4b70-ba5d-75899ed8ee4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-49335d47-e0ec-4b70-ba5d-75899ed8ee4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:21:38.903830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:21:38.905085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:21:38.906274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 596/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:21:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:21:38.911547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc06e56-80b8-43b1-b127-ca11d3c1c220 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:38.965075 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:39.019486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc06e56-80b8-43b1-b127-ca11d3c1c220 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:39.053735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6ecb5-1e47-461f-a48b-5f3c3c15d3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:39.054430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6ecb5-1e47-461f-a48b-5f3c3c15d3ea None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:39.054710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6ecb5-1e47-461f-a48b-5f3c3c15d3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:39.054982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6ecb5-1e47-461f-a48b-5f3c3c15d3ea None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:39.055643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6ecb5-1e47-461f-a48b-5f3c3c15d3ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:39.055945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6ecb5-1e47-461f-a48b-5f3c3c15d3ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:39.057331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6ecb5-1e47-461f-a48b-5f3c3c15d3ea None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:39.065129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 600/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:39.071104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49335d47-e0ec-4b70-ba5d-75899ed8ee4c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:39.167601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49335d47-e0ec-4b70-ba5d-75899ed8ee4c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:39.199720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf972a-1814-4eef-a935-45e2f68648a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:39.200351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf972a-1814-4eef-a935-45e2f68648a8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:39.200627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf972a-1814-4eef-a935-45e2f68648a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:39.200895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf972a-1814-4eef-a935-45e2f68648a8 None admin] PATH_INFO: `/v3/users/5384e6c51181428cab0c2e9aeca4b38c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:39.202000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf972a-1814-4eef-a935-45e2f68648a8 None admin] RBAC: Authorizing `identity:get_user(user_id=5384e6c51181428cab0c2e9aeca4b38c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:39.203404 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf972a-1814-4eef-a935-45e2f68648a8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:39.204964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 597/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 22:21:39 2022] GET /identity/v3/users/5384e6c51181428cab0c2e9aeca4b38c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:39.211317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6ecb5-1e47-461f-a48b-5f3c3c15d3ea None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:39.308021 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6ecb5-1e47-461f-a48b-5f3c3c15d3ea None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:39.316715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:39.341495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6e1e6-42cd-43d1-800a-89b249928cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:39.342087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6e1e6-42cd-43d1-800a-89b249928cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:39.342363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6e1e6-42cd-43d1-800a-89b249928cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:39.342633 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6e1e6-42cd-43d1-800a-89b249928cf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:39.406873 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6e1e6-42cd-43d1-800a-89b249928cf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:39.408330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6e1e6-42cd-43d1-800a-89b249928cf8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:39.500763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 601/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:21:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:21:39.506600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf972a-1814-4eef-a935-45e2f68648a8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:39.537693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:39.603652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf972a-1814-4eef-a935-45e2f68648a8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:39.637503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d492b-48db-47e9-b9bc-d7e4644fe176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:39.638108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d492b-48db-47e9-b9bc-d7e4644fe176 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:39.638383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d492b-48db-47e9-b9bc-d7e4644fe176 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:39.638655 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d492b-48db-47e9-b9bc-d7e4644fe176 None admin] PATH_INFO: `/v3/projects/34472727bdad4c0496400ed82ab60cb1` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:39.639766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d492b-48db-47e9-b9bc-d7e4644fe176 None admin] RBAC: Authorizing `identity:get_project(project_id=34472727bdad4c0496400ed82ab60cb1)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:39.641205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d492b-48db-47e9-b9bc-d7e4644fe176 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:39.648158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 598/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 22:21:39 2022] GET /identity/v3/projects/34472727bdad4c0496400ed82ab60cb1 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:39.649052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6e1e6-42cd-43d1-800a-89b249928cf8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:21:39.658892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:39.744025 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6e1e6-42cd-43d1-800a-89b249928cf8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:21:39.776452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2331aa-5199-42a2-bc27-2ebb1cc9c867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:21:39.777035 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2331aa-5199-42a2-bc27-2ebb1cc9c867 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:21:39.777312 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2331aa-5199-42a2-bc27-2ebb1cc9c867 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:21:39.777578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2331aa-5199-42a2-bc27-2ebb1cc9c867 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:21:39.778289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2331aa-5199-42a2-bc27-2ebb1cc9c867 None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=34472727bdad4c0496400ed82ab60cb1, user.id=5384e6c51181428cab0c2e9aeca4b38c) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:21:39.778589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2331aa-5199-42a2-bc27-2ebb1cc9c867 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:21:39.780007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2331aa-5199-42a2-bc27-2ebb1cc9c867 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:21:39.785512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 602/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:21:39 2022] GET /identity/v3/role_assignments?user.id=5384e6c51181428cab0c2e9aeca4b38c&scope.project.id=34472727bdad4c0496400ed82ab60cb1&role.id=108664305d2546949f7327b65d4cb667 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:21:49.633439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=50 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:21:49.783024 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=64 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:51.480702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce393307-84fd-48c5-8f3e-b2b1cdc831ab None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:51.481311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce393307-84fd-48c5-8f3e-b2b1cdc831ab None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:51.482064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce393307-84fd-48c5-8f3e-b2b1cdc831ab None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:51.483375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 599/1201] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:22:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:22:51.488431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e10045-436c-4cea-a77a-57d6a24f28b9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:51.488880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e10045-436c-4cea-a77a-57d6a24f28b9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:51.488981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e10045-436c-4cea-a77a-57d6a24f28b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:51.507974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-44e10045-436c-4cea-a77a-57d6a24f28b9 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:22:51.510481 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-44e10045-436c-4cea-a77a-57d6a24f28b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:22:51.573538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 603/1202] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:22:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:22:51.598188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce393307-84fd-48c5-8f3e-b2b1cdc831ab None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:51.698402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce393307-84fd-48c5-8f3e-b2b1cdc831ab None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:51.732284 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f11c10-a953-4821-9737-d43cf995d7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:51.732830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11c10-a953-4821-9737-d43cf995d7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:51.733105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11c10-a953-4821-9737-d43cf995d7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:51.733379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11c10-a953-4821-9737-d43cf995d7b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:51.743106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f11c10-a953-4821-9737-d43cf995d7b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:51.744556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f11c10-a953-4821-9737-d43cf995d7b9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-96f11c10-a953-4821-9737-d43cf995d7b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:22:51.759142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:22:51.760843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:22:51.762173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:22:51.762173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:22:51.762173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:22:51.762173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 600/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:22:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:22:51.767009 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e10045-436c-4cea-a77a-57d6a24f28b9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:51.868736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e10045-436c-4cea-a77a-57d6a24f28b9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:51.897768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:51.903502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416899f-caec-4d5d-a97b-3f6bed2b53ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:51.904231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416899f-caec-4d5d-a97b-3f6bed2b53ca None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:51.904509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416899f-caec-4d5d-a97b-3f6bed2b53ca None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:51.904779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416899f-caec-4d5d-a97b-3f6bed2b53ca None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:51.905421 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416899f-caec-4d5d-a97b-3f6bed2b53ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:51.905724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416899f-caec-4d5d-a97b-3f6bed2b53ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:51.907090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416899f-caec-4d5d-a97b-3f6bed2b53ca None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:51.918197 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 604/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:22:51.924453 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f11c10-a953-4821-9737-d43cf995d7b9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:51.977321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:52.023058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f11c10-a953-4821-9737-d43cf995d7b9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:52.055357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b99ef4-a729-463d-a097-35cbe95c800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:52.055949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b99ef4-a729-463d-a097-35cbe95c800c None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:52.056251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b99ef4-a729-463d-a097-35cbe95c800c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:52.056526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b99ef4-a729-463d-a097-35cbe95c800c None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:52.057163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b99ef4-a729-463d-a097-35cbe95c800c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:52.058568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b99ef4-a729-463d-a097-35cbe95c800c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:52.140957 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 601/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 22:22:51 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:22:54.149474 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4453-5561-4556-bd47-14a1412ef936 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:54.150021 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4453-5561-4556-bd47-14a1412ef936 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:54.150125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4453-5561-4556-bd47-14a1412ef936 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:54.151551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 605/1206] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:22:54.157519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbb6d6-4b39-4d00-97ff-7221b09cf0df None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:54.157940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbb6d6-4b39-4d00-97ff-7221b09cf0df None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:54.158079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbb6d6-4b39-4d00-97ff-7221b09cf0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:54.176326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e3bbb6d6-4b39-4d00-97ff-7221b09cf0df None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:22:54.177513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e3bbb6d6-4b39-4d00-97ff-7221b09cf0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:22:54.243789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 602/1207] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:22:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:22:54.267253 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd4453-5561-4556-bd47-14a1412ef936 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:54.370475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd4453-5561-4556-bd47-14a1412ef936 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:54.405328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9de29-c36f-42b7-a0d6-e597082da3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:54.405948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9de29-c36f-42b7-a0d6-e597082da3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:54.406226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9de29-c36f-42b7-a0d6-e597082da3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:54.406507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9de29-c36f-42b7-a0d6-e597082da3b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:54.416448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9de29-c36f-42b7-a0d6-e597082da3b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-62c9de29-c36f-42b7-a0d6-e597082da3b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:22:54.421181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:22:54.422296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:22:54.423326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 606/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:22:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:22:54.428605 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbb6d6-4b39-4d00-97ff-7221b09cf0df None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:54.468709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:54.527341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbb6d6-4b39-4d00-97ff-7221b09cf0df None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:54.561083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29eadb-e383-4a20-9cd5-012ab5fe89ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:54.561754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29eadb-e383-4a20-9cd5-012ab5fe89ce None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:54.561968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29eadb-e383-4a20-9cd5-012ab5fe89ce None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:54.562259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29eadb-e383-4a20-9cd5-012ab5fe89ce None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:54.562915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29eadb-e383-4a20-9cd5-012ab5fe89ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:54.563216 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29eadb-e383-4a20-9cd5-012ab5fe89ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:54.564611 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29eadb-e383-4a20-9cd5-012ab5fe89ce None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:54.572303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 603/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:22:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:22:54.580773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9de29-c36f-42b7-a0d6-e597082da3b7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:54.629926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:54.677800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9de29-c36f-42b7-a0d6-e597082da3b7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:54.713166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc734e-fa97-4600-8ea4-ee10fdbad6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:54.713785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc734e-fa97-4600-8ea4-ee10fdbad6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:54.714061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc734e-fa97-4600-8ea4-ee10fdbad6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:54.714329 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc734e-fa97-4600-8ea4-ee10fdbad6c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:54.725693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc734e-fa97-4600-8ea4-ee10fdbad6c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:54.727148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc734e-fa97-4600-8ea4-ee10fdbad6c1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-17fc734e-fa97-4600-8ea4-ee10fdbad6c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:22:54.741539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:22:54.742701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:22:54.744126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:22:54.744126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:22:54.744126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:22:54.744126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 607/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:22:54.749423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29eadb-e383-4a20-9cd5-012ab5fe89ce None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:54.847588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:54.850438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29eadb-e383-4a20-9cd5-012ab5fe89ce None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:54.884396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687510e-1b9a-445b-9d17-38e6e3715bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:54.884981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687510e-1b9a-445b-9d17-38e6e3715bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:54.885258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687510e-1b9a-445b-9d17-38e6e3715bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:54.885533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687510e-1b9a-445b-9d17-38e6e3715bb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:54.886163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687510e-1b9a-445b-9d17-38e6e3715bb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:54.886462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687510e-1b9a-445b-9d17-38e6e3715bb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:54.887841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687510e-1b9a-445b-9d17-38e6e3715bb3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:54.898499 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 604/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:22:54.904513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc734e-fa97-4600-8ea4-ee10fdbad6c1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:55.006545 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc734e-fa97-4600-8ea4-ee10fdbad6c1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:55.008526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:55.040923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee8e7d-3510-4ff3-8766-ad95ee520b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:55.041546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee8e7d-3510-4ff3-8766-ad95ee520b87 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:55.041818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee8e7d-3510-4ff3-8766-ad95ee520b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:55.042084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee8e7d-3510-4ff3-8766-ad95ee520b87 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:55.049722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee8e7d-3510-4ff3-8766-ad95ee520b87 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:55.051162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee8e7d-3510-4ff3-8766-ad95ee520b87 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-91ee8e7d-3510-4ff3-8766-ad95ee520b87 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:22:55.062830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:22:55.063973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:22:55.065183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:22:55.065183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:22:55.065183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:22:55.065183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:22:55.065183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:22:55.065183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:22:55.065183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:22:55.065183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:22:55.065183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:22:55.065183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:22:55.065183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 608/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:22:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:22:55.069692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687510e-1b9a-445b-9d17-38e6e3715bb3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:55.168426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687510e-1b9a-445b-9d17-38e6e3715bb3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:55.200620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb91b1-7d83-4ac0-90e8-7d0fbcd50ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:55.201210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb91b1-7d83-4ac0-90e8-7d0fbcd50ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:55.201488 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb91b1-7d83-4ac0-90e8-7d0fbcd50ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:55.201760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb91b1-7d83-4ac0-90e8-7d0fbcd50ee7 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:55.202475 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb91b1-7d83-4ac0-90e8-7d0fbcd50ee7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:55.202780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb91b1-7d83-4ac0-90e8-7d0fbcd50ee7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:55.204211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb91b1-7d83-4ac0-90e8-7d0fbcd50ee7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:55.221255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 605/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 22:22:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:22:55.227129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee8e7d-3510-4ff3-8766-ad95ee520b87 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:55.329490 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee8e7d-3510-4ff3-8766-ad95ee520b87 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:55.363739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe0c94-767f-4b9f-be8a-09d49c21ccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:55.364370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe0c94-767f-4b9f-be8a-09d49c21ccd6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:55.364652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe0c94-767f-4b9f-be8a-09d49c21ccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:55.364928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe0c94-767f-4b9f-be8a-09d49c21ccd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:55.394131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:55.431759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe0c94-767f-4b9f-be8a-09d49c21ccd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:55.433351 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe0c94-767f-4b9f-be8a-09d49c21ccd6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:55.529968 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 609/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:22:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:22:55.535765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb91b1-7d83-4ac0-90e8-7d0fbcd50ee7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:55.554501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:55.634822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb91b1-7d83-4ac0-90e8-7d0fbcd50ee7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:55.669631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d328598-b27a-4624-9980-b8060baae73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:55.670249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d328598-b27a-4624-9980-b8060baae73c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:55.670532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d328598-b27a-4624-9980-b8060baae73c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:55.670821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d328598-b27a-4624-9980-b8060baae73c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:55.680782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d328598-b27a-4624-9980-b8060baae73c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:55.687621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d328598-b27a-4624-9980-b8060baae73c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-4d328598-b27a-4624-9980-b8060baae73c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:22:55.696657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:22:55.697773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:22:55.699092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:22:55.699092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:22:55.699092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:22:55.699092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 606/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:22:55.704117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe0c94-767f-4b9f-be8a-09d49c21ccd6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:55.765744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:55.803881 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe0c94-767f-4b9f-be8a-09d49c21ccd6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:55.836472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d7df3-6cdc-42c0-b2af-753f49271a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:55.837069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d7df3-6cdc-42c0-b2af-753f49271a7b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:55.837348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d7df3-6cdc-42c0-b2af-753f49271a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:55.837618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d7df3-6cdc-42c0-b2af-753f49271a7b None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:55.838251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d7df3-6cdc-42c0-b2af-753f49271a7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:55.838556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d7df3-6cdc-42c0-b2af-753f49271a7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:55.839923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d7df3-6cdc-42c0-b2af-753f49271a7b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:55.852353 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 610/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:22:55.858896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d328598-b27a-4624-9980-b8060baae73c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:55.926816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:55.956005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d328598-b27a-4624-9980-b8060baae73c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:55.989521 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dca883-4f84-4dd0-8604-e21d74759e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:55.990169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dca883-4f84-4dd0-8604-e21d74759e28 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:55.990449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dca883-4f84-4dd0-8604-e21d74759e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:55.990726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dca883-4f84-4dd0-8604-e21d74759e28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:55.997319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dca883-4f84-4dd0-8604-e21d74759e28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:56.001547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dca883-4f84-4dd0-8604-e21d74759e28 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c7dca883-4f84-4dd0-8604-e21d74759e28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:22:56.009917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:22:56.010825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:22:56.011793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:22:56.011793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:22:56.011793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:22:56.011793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:22:56.011793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:22:56.011793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:22:56.011793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:22:56.011793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:22:56.011793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 607/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:22:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:22:56.017084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d7df3-6cdc-42c0-b2af-753f49271a7b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:56.116547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d7df3-6cdc-42c0-b2af-753f49271a7b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:56.138782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:56.149635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03494a89-b3ad-46cd-8d46-0b4c574b2750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:56.150228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494a89-b3ad-46cd-8d46-0b4c574b2750 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:56.150501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494a89-b3ad-46cd-8d46-0b4c574b2750 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:56.150824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494a89-b3ad-46cd-8d46-0b4c574b2750 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:56.151519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03494a89-b3ad-46cd-8d46-0b4c574b2750 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:56.151822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03494a89-b3ad-46cd-8d46-0b4c574b2750 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:56.153940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03494a89-b3ad-46cd-8d46-0b4c574b2750 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:56.165007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 611/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:22:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:22:56.171984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dca883-4f84-4dd0-8604-e21d74759e28 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:56.269867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dca883-4f84-4dd0-8604-e21d74759e28 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:56.297120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:56.302871 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bba23-f76d-4bf5-9950-1c1c11a54b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:56.303469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bba23-f76d-4bf5-9950-1c1c11a54b8e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:56.303750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bba23-f76d-4bf5-9950-1c1c11a54b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:56.304034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bba23-f76d-4bf5-9950-1c1c11a54b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:56.304769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bba23-f76d-4bf5-9950-1c1c11a54b8e None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=db1d7e39e8c14f5db40efa743fc3b652) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:56.305128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bba23-f76d-4bf5-9950-1c1c11a54b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:56.306531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bba23-f76d-4bf5-9950-1c1c11a54b8e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:56.311966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 608/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:22:56 2022] GET /identity/v3/role_assignments?user.id=db1d7e39e8c14f5db40efa743fc3b652&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:22:58.281943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addf39f-5061-465c-9d0a-beb3ea6fea08 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:58.282466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addf39f-5061-465c-9d0a-beb3ea6fea08 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:58.282581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addf39f-5061-465c-9d0a-beb3ea6fea08 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:58.283864 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 612/1220] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:22:58.288913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302de21e-f109-40d8-a72c-e79ce18edbaf None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:58.289322 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302de21e-f109-40d8-a72c-e79ce18edbaf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:58.291947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302de21e-f109-40d8-a72c-e79ce18edbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:58.307525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-302de21e-f109-40d8-a72c-e79ce18edbaf None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:22:58.308785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-302de21e-f109-40d8-a72c-e79ce18edbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:22:58.376157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 609/1221] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:22:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:22:58.398691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0addf39f-5061-465c-9d0a-beb3ea6fea08 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:58.505586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0addf39f-5061-465c-9d0a-beb3ea6fea08 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:58.539432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2b66e-edb9-4009-9924-4ebd1bd3180b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:58.540057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2b66e-edb9-4009-9924-4ebd1bd3180b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:58.540341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2b66e-edb9-4009-9924-4ebd1bd3180b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:58.540614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2b66e-edb9-4009-9924-4ebd1bd3180b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:58.551119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2b66e-edb9-4009-9924-4ebd1bd3180b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-0bc2b66e-edb9-4009-9924-4ebd1bd3180b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:22:58.555841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:22:58.558848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:22:58.560647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 613/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:22:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:22:58.563839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302de21e-f109-40d8-a72c-e79ce18edbaf None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:58.661686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302de21e-f109-40d8-a72c-e79ce18edbaf None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:58.685951 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:58.696378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b515f-af4b-41ca-b9ac-96588d9df92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:58.696978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b515f-af4b-41ca-b9ac-96588d9df92b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:58.697255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b515f-af4b-41ca-b9ac-96588d9df92b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:58.697571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b515f-af4b-41ca-b9ac-96588d9df92b None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:58.698232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b515f-af4b-41ca-b9ac-96588d9df92b None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:58.698533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b515f-af4b-41ca-b9ac-96588d9df92b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:58.700091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b515f-af4b-41ca-b9ac-96588d9df92b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:58.707729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 610/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:22:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:22:58.713572 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2b66e-edb9-4009-9924-4ebd1bd3180b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:58.757952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:58.813537 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2b66e-edb9-4009-9924-4ebd1bd3180b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:58.847959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f69c08-e551-447b-82b9-53a872aed5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:58.848669 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f69c08-e551-447b-82b9-53a872aed5da None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:58.848943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f69c08-e551-447b-82b9-53a872aed5da None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:58.849213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f69c08-e551-447b-82b9-53a872aed5da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:58.859911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f69c08-e551-447b-82b9-53a872aed5da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:58.861374 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f69c08-e551-447b-82b9-53a872aed5da None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-e5f69c08-e551-447b-82b9-53a872aed5da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:22:58.876105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:22:58.877343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:22:58.878412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:22:58.878412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:22:58.878412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:22:58.878412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 614/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:22:58.883115 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b515f-af4b-41ca-b9ac-96588d9df92b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:58.986207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b515f-af4b-41ca-b9ac-96588d9df92b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:59.019131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc2ff8-c177-492f-8be1-a53b54952908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:59.019744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc2ff8-c177-492f-8be1-a53b54952908 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:59.020037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc2ff8-c177-492f-8be1-a53b54952908 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:59.020316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc2ff8-c177-492f-8be1-a53b54952908 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:59.020947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc2ff8-c177-492f-8be1-a53b54952908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:59.021245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc2ff8-c177-492f-8be1-a53b54952908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:59.022585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc2ff8-c177-492f-8be1-a53b54952908 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:59.034552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 611/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:22:59.040312 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f69c08-e551-447b-82b9-53a872aed5da None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:59.134894 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:59.141521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f69c08-e551-447b-82b9-53a872aed5da None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:59.175971 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e248d-04df-4642-b909-b2022e9fa45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:59.176648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e248d-04df-4642-b909-b2022e9fa45a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:59.176933 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e248d-04df-4642-b909-b2022e9fa45a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:59.177242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e248d-04df-4642-b909-b2022e9fa45a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:59.184834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e248d-04df-4642-b909-b2022e9fa45a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:59.186327 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e248d-04df-4642-b909-b2022e9fa45a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-858e248d-04df-4642-b909-b2022e9fa45a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:22:59.198128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:22:59.199265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:22:59.201080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:22:59.201080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:22:59.201080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:22:59.201080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:22:59.201080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:22:59.201080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:22:59.201080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:22:59.201080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:22:59.201080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:22:59.201080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:22:59.201080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 615/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:22:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:22:59.205807 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc2ff8-c177-492f-8be1-a53b54952908 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:59.254413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:59.306251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc2ff8-c177-492f-8be1-a53b54952908 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:59.339663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccdb0a-a154-4d9b-baa9-ba9e37b3a7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:59.340893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccdb0a-a154-4d9b-baa9-ba9e37b3a7ab None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:59.341182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccdb0a-a154-4d9b-baa9-ba9e37b3a7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:59.341454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccdb0a-a154-4d9b-baa9-ba9e37b3a7ab None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:59.342167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccdb0a-a154-4d9b-baa9-ba9e37b3a7ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:59.342471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccdb0a-a154-4d9b-baa9-ba9e37b3a7ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:59.343879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccdb0a-a154-4d9b-baa9-ba9e37b3a7ab None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:59.360666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 612/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 22:22:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:22:59.366562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e248d-04df-4642-b909-b2022e9fa45a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:59.468870 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e248d-04df-4642-b909-b2022e9fa45a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:59.502518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f783bb6-5333-471e-af97-8b7d0be05fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:59.503138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f783bb6-5333-471e-af97-8b7d0be05fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:59.503413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f783bb6-5333-471e-af97-8b7d0be05fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:59.503685 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f783bb6-5333-471e-af97-8b7d0be05fa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:59.514691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f783bb6-5333-471e-af97-8b7d0be05fa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:59.516190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f783bb6-5333-471e-af97-8b7d0be05fa4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-9f783bb6-5333-471e-af97-8b7d0be05fa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:22:59.530951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:22:59.532189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:22:59.533456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:22:59.533456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:22:59.533456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:22:59.533456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 616/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:22:59.537864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccdb0a-a154-4d9b-baa9-ba9e37b3a7ab None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:59.645029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccdb0a-a154-4d9b-baa9-ba9e37b3a7ab None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:59.647311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:59.678463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e791d-e9d9-44e6-b0c7-18226f9fd71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:59.679060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e791d-e9d9-44e6-b0c7-18226f9fd71a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:59.679340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e791d-e9d9-44e6-b0c7-18226f9fd71a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:59.679617 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e791d-e9d9-44e6-b0c7-18226f9fd71a None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:59.680287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e791d-e9d9-44e6-b0c7-18226f9fd71a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:59.680595 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e791d-e9d9-44e6-b0c7-18226f9fd71a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:59.681952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e791d-e9d9-44e6-b0c7-18226f9fd71a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:59.693456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 613/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:22:59.699150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f783bb6-5333-471e-af97-8b7d0be05fa4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:59.709192 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:22:59.798404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f783bb6-5333-471e-af97-8b7d0be05fa4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:59.831877 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1df25-bbb5-47ca-bad5-a01c1b2d99fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:59.832524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1df25-bbb5-47ca-bad5-a01c1b2d99fe None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:59.832805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1df25-bbb5-47ca-bad5-a01c1b2d99fe None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:59.833079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1df25-bbb5-47ca-bad5-a01c1b2d99fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:59.839566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1df25-bbb5-47ca-bad5-a01c1b2d99fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:22:59.841159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1df25-bbb5-47ca-bad5-a01c1b2d99fe None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-34e1df25-bbb5-47ca-bad5-a01c1b2d99fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:22:59.852323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:22:59.853358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:22:59.854700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:22:59.854700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:22:59.854700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:22:59.854700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:22:59.854700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:22:59.854700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:22:59.854700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:22:59.854700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:22:59.854700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 617/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:22:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:22:59.859304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e791d-e9d9-44e6-b0c7-18226f9fd71a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:22:59.963015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e791d-e9d9-44e6-b0c7-18226f9fd71a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:22:59.996143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af02c93-0953-4a2d-9ec6-c65ddd04f606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:22:59.996801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af02c93-0953-4a2d-9ec6-c65ddd04f606 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:22:59.997078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af02c93-0953-4a2d-9ec6-c65ddd04f606 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:22:59.997349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af02c93-0953-4a2d-9ec6-c65ddd04f606 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:22:59.998283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af02c93-0953-4a2d-9ec6-c65ddd04f606 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:22:59.998598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af02c93-0953-4a2d-9ec6-c65ddd04f606 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:00.000559 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af02c93-0953-4a2d-9ec6-c65ddd04f606 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:00.012261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 614/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:22:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:00.018363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1df25-bbb5-47ca-bad5-a01c1b2d99fe None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:00.083312 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:00.120296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1df25-bbb5-47ca-bad5-a01c1b2d99fe None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:00.154695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea85b0a-5fb0-44cc-9882-f52f24f816c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:00.155341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea85b0a-5fb0-44cc-9882-f52f24f816c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:00.155615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea85b0a-5fb0-44cc-9882-f52f24f816c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:00.155887 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea85b0a-5fb0-44cc-9882-f52f24f816c3 None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/db1d7e39e8c14f5db40efa743fc3b652/roles/900062fd06bc4f00add25eb39325653f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:00.170229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea85b0a-5fb0-44cc-9882-f52f24f816c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=db1d7e39e8c14f5db40efa743fc3b652, role_id=900062fd06bc4f00add25eb39325653f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:00.171657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea85b0a-5fb0-44cc-9882-f52f24f816c3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:00.242803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 618/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:23:00 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/db1d7e39e8c14f5db40efa743fc3b652/roles/900062fd06bc4f00add25eb39325653f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:23:02.249009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088f528-c0fb-461f-8394-ab67b2e7f9dc None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:02.249525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088f528-c0fb-461f-8394-ab67b2e7f9dc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:02.249525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088f528-c0fb-461f-8394-ab67b2e7f9dc None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:02.250786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 615/1233] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:23:02.255788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d21d2-f26b-4df0-85d9-6f7b67000d88 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:02.256231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d21d2-f26b-4df0-85d9-6f7b67000d88 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:02.256366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d21d2-f26b-4df0-85d9-6f7b67000d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:02.275440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-314d21d2-f26b-4df0-85d9-6f7b67000d88 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:23:02.276692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-314d21d2-f26b-4df0-85d9-6f7b67000d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:23:02.343580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 619/1234] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:23:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:23:02.367385 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088f528-c0fb-461f-8394-ab67b2e7f9dc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:02.392834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:02.467785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088f528-c0fb-461f-8394-ab67b2e7f9dc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:02.501391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9feb-861c-45fe-9c39-9ad45c0b24ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:02.501947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9feb-861c-45fe-9c39-9ad45c0b24ce None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:02.502326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9feb-861c-45fe-9c39-9ad45c0b24ce None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:02.502619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9feb-861c-45fe-9c39-9ad45c0b24ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:02.514181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a9feb-861c-45fe-9c39-9ad45c0b24ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-be6a9feb-861c-45fe-9c39-9ad45c0b24ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:23:02.518924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:02.521203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:02.522393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 616/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 22:23:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:02.526141 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d21d2-f26b-4df0-85d9-6f7b67000d88 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:02.573082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:02.628603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d21d2-f26b-4df0-85d9-6f7b67000d88 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:02.661501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e92213-4a80-4dbd-8219-e8b2e87a4b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:02.662101 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e92213-4a80-4dbd-8219-e8b2e87a4b7f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:02.662378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e92213-4a80-4dbd-8219-e8b2e87a4b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:02.662650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e92213-4a80-4dbd-8219-e8b2e87a4b7f None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:02.663314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e92213-4a80-4dbd-8219-e8b2e87a4b7f None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:02.663618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e92213-4a80-4dbd-8219-e8b2e87a4b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:02.665038 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e92213-4a80-4dbd-8219-e8b2e87a4b7f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:02.672713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 620/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 22:23:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:02.678569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9feb-861c-45fe-9c39-9ad45c0b24ce None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:02.762798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:02.778388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9feb-861c-45fe-9c39-9ad45c0b24ce None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:02.811919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e31439-811b-461f-a2cf-8fcb6693480c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:02.812572 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e31439-811b-461f-a2cf-8fcb6693480c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:02.812867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e31439-811b-461f-a2cf-8fcb6693480c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:02.813141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e31439-811b-461f-a2cf-8fcb6693480c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:02.824235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e31439-811b-461f-a2cf-8fcb6693480c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:02.826307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e31439-811b-461f-a2cf-8fcb6693480c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f1e31439-811b-461f-a2cf-8fcb6693480c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:23:02.840727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:02.841684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:23:02.842621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:23:02.842621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:02.842621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:02.842621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 617/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:23:02.847681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e92213-4a80-4dbd-8219-e8b2e87a4b7f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:02.946039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e92213-4a80-4dbd-8219-e8b2e87a4b7f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:02.956136 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:02.980391 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f3873-295e-4588-b56f-ad916805f787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:02.981767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f3873-295e-4588-b56f-ad916805f787 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:02.982123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f3873-295e-4588-b56f-ad916805f787 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:02.982396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f3873-295e-4588-b56f-ad916805f787 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:02.983031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f3873-295e-4588-b56f-ad916805f787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:02.983334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f3873-295e-4588-b56f-ad916805f787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:02.984718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f3873-295e-4588-b56f-ad916805f787 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:02.997004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 621/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:03.003018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e31439-811b-461f-a2cf-8fcb6693480c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:03.101194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e31439-811b-461f-a2cf-8fcb6693480c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:03.134678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d225a-0a9e-460b-b17b-d75a5f3f7a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:03.135334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d225a-0a9e-460b-b17b-d75a5f3f7a40 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:03.135608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d225a-0a9e-460b-b17b-d75a5f3f7a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:03.135878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d225a-0a9e-460b-b17b-d75a5f3f7a40 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:03.138660 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:03.143602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d225a-0a9e-460b-b17b-d75a5f3f7a40 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:03.145107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d225a-0a9e-460b-b17b-d75a5f3f7a40 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f20d225a-0a9e-460b-b17b-d75a5f3f7a40 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:23:03.155920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:03.156972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:03.158483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:03.158483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:03.158483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:23:03.158483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:23:03.158483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:23:03.158483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:23:03.158483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:23:03.158483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:23:03.158483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:23:03.158483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:03.158483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 618/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:23:03.162867 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f3873-295e-4588-b56f-ad916805f787 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:03.260868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f3873-295e-4588-b56f-ad916805f787 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:03.293831 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15566b7-07d4-4058-bf04-2f8a0c3e5f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:03.294433 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15566b7-07d4-4058-bf04-2f8a0c3e5f23 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:03.294711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15566b7-07d4-4058-bf04-2f8a0c3e5f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:03.294985 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15566b7-07d4-4058-bf04-2f8a0c3e5f23 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:03.295706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15566b7-07d4-4058-bf04-2f8a0c3e5f23 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:03.296031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15566b7-07d4-4058-bf04-2f8a0c3e5f23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:03.297472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15566b7-07d4-4058-bf04-2f8a0c3e5f23 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:03.314606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 622/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 22:23:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:03.320442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d225a-0a9e-460b-b17b-d75a5f3f7a40 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:03.420449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d225a-0a9e-460b-b17b-d75a5f3f7a40 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:03.453082 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab127d10-7870-46e7-9af1-0269432bba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:03.453828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127d10-7870-46e7-9af1-0269432bba9c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:03.454110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127d10-7870-46e7-9af1-0269432bba9c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:03.454399 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127d10-7870-46e7-9af1-0269432bba9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:03.515826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:03.525780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab127d10-7870-46e7-9af1-0269432bba9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:03.527334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab127d10-7870-46e7-9af1-0269432bba9c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:03.630970 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 619/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:23:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:23:03.636813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15566b7-07d4-4058-bf04-2f8a0c3e5f23 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:03.665737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:03.737545 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15566b7-07d4-4058-bf04-2f8a0c3e5f23 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:03.771893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad5d8b-7077-48a1-972a-0a5fd814bded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:03.772547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad5d8b-7077-48a1-972a-0a5fd814bded None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:03.772831 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad5d8b-7077-48a1-972a-0a5fd814bded None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:03.773105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad5d8b-7077-48a1-972a-0a5fd814bded None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:03.782884 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad5d8b-7077-48a1-972a-0a5fd814bded None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:03.784370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad5d8b-7077-48a1-972a-0a5fd814bded None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-e5ad5d8b-7077-48a1-972a-0a5fd814bded None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:23:03.800872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:03.802105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:23:03.803379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:23:03.803379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:03.803379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:03.803379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 623/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:03.807798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab127d10-7870-46e7-9af1-0269432bba9c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:03.900163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:03.906233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab127d10-7870-46e7-9af1-0269432bba9c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:03.940047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4ffdc-76b4-4259-9e51-cf9fa99cfc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:03.940302 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ffdc-76b4-4259-9e51-cf9fa99cfc26 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:03.940483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ffdc-76b4-4259-9e51-cf9fa99cfc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:03.940761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ffdc-76b4-4259-9e51-cf9fa99cfc26 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:03.941403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4ffdc-76b4-4259-9e51-cf9fa99cfc26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:03.941717 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4ffdc-76b4-4259-9e51-cf9fa99cfc26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:03.943072 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4ffdc-76b4-4259-9e51-cf9fa99cfc26 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:03.954440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 620/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:03.960475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad5d8b-7077-48a1-972a-0a5fd814bded None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:04.040333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:04.061904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad5d8b-7077-48a1-972a-0a5fd814bded None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:04.093762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef30f7-8737-4678-b07e-8f33be48d0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:04.094370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef30f7-8737-4678-b07e-8f33be48d0de None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:04.094646 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef30f7-8737-4678-b07e-8f33be48d0de None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:04.094917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef30f7-8737-4678-b07e-8f33be48d0de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:04.101345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef30f7-8737-4678-b07e-8f33be48d0de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:04.102701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef30f7-8737-4678-b07e-8f33be48d0de None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f8ef30f7-8737-4678-b07e-8f33be48d0de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:23:04.113989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:04.115052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:04.116119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:23:04.116119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:23:04.116119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:23:04.116119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:23:04.116119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:23:04.116119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:23:04.116119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:23:04.116119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:04.116119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 624/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:23:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:23:04.121500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4ffdc-76b4-4259-9e51-cf9fa99cfc26 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:04.220549 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4ffdc-76b4-4259-9e51-cf9fa99cfc26 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:04.253446 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899937e4-6984-49db-a58a-c01af490aab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:04.254041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899937e4-6984-49db-a58a-c01af490aab7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:04.254319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899937e4-6984-49db-a58a-c01af490aab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:04.254592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899937e4-6984-49db-a58a-c01af490aab7 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:04.255271 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899937e4-6984-49db-a58a-c01af490aab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:04.255582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899937e4-6984-49db-a58a-c01af490aab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:04.257022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899937e4-6984-49db-a58a-c01af490aab7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:04.268618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 621/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:23:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:04.274784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef30f7-8737-4678-b07e-8f33be48d0de None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:04.375002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef30f7-8737-4678-b07e-8f33be48d0de None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:04.408186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442f02e-cd68-4897-92b6-fa20c4b5f720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:04.408780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f02e-cd68-4897-92b6-fa20c4b5f720 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:04.409063 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f02e-cd68-4897-92b6-fa20c4b5f720 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:04.409338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f02e-cd68-4897-92b6-fa20c4b5f720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:04.410056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442f02e-cd68-4897-92b6-fa20c4b5f720 None admin] RBAC: Adding query filter params (role.id=900062fd06bc4f00add25eb39325653f, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=db1d7e39e8c14f5db40efa743fc3b652) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:04.410363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442f02e-cd68-4897-92b6-fa20c4b5f720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:04.411778 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442f02e-cd68-4897-92b6-fa20c4b5f720 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:04.417373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 625/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:23:04 2022] GET /identity/v3/role_assignments?user.id=db1d7e39e8c14f5db40efa743fc3b652&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=900062fd06bc4f00add25eb39325653f => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:06.479849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d6765-55d3-4a6f-b3ce-8459829d89d9 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:06.480468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d6765-55d3-4a6f-b3ce-8459829d89d9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:06.480468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d6765-55d3-4a6f-b3ce-8459829d89d9 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:06.482357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 622/1247] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:23:06.487560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a2e4-fe20-48ea-9d13-b25471b448a2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:06.491222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a2e4-fe20-48ea-9d13-b25471b448a2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:06.491222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a2e4-fe20-48ea-9d13-b25471b448a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:06.494663 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:06.507248 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-09a9a2e4-fe20-48ea-9d13-b25471b448a2 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:23:06.508491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-09a9a2e4-fe20-48ea-9d13-b25471b448a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:23:06.574879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 626/1248] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:23:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:23:06.599234 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d6765-55d3-4a6f-b3ce-8459829d89d9 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:06.617789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:06.701886 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d6765-55d3-4a6f-b3ce-8459829d89d9 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:06.735131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8110e-915c-4bf2-bee0-03d7c880086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:06.735682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8110e-915c-4bf2-bee0-03d7c880086e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:06.735960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8110e-915c-4bf2-bee0-03d7c880086e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:06.736338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8110e-915c-4bf2-bee0-03d7c880086e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:06.747550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8110e-915c-4bf2-bee0-03d7c880086e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-34a8110e-915c-4bf2-bee0-03d7c880086e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:23:06.752786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:06.754006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:06.755260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 623/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:23:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:06.760113 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9a2e4-fe20-48ea-9d13-b25471b448a2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:06.858915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9a2e4-fe20-48ea-9d13-b25471b448a2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:06.888074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:06.893035 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f50ac-0afa-4e11-93e4-f47408c1d77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:06.893643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f50ac-0afa-4e11-93e4-f47408c1d77f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:06.893920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f50ac-0afa-4e11-93e4-f47408c1d77f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:06.894193 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f50ac-0afa-4e11-93e4-f47408c1d77f None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:06.894857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f50ac-0afa-4e11-93e4-f47408c1d77f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:06.895162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f50ac-0afa-4e11-93e4-f47408c1d77f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:06.896558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f50ac-0afa-4e11-93e4-f47408c1d77f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:06.905133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 627/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:23:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:06.910916 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8110e-915c-4bf2-bee0-03d7c880086e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:06.988520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:07.010349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8110e-915c-4bf2-bee0-03d7c880086e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:07.045886 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4eb2e-0bcd-4906-8009-c5cebc59c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:07.046506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4eb2e-0bcd-4906-8009-c5cebc59c7bf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:07.046786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4eb2e-0bcd-4906-8009-c5cebc59c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:07.047058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4eb2e-0bcd-4906-8009-c5cebc59c7bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:07.056926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4eb2e-0bcd-4906-8009-c5cebc59c7bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:07.058371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4eb2e-0bcd-4906-8009-c5cebc59c7bf None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-04e4eb2e-0bcd-4906-8009-c5cebc59c7bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:23:07.072878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:07.073984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:23:07.075815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:23:07.075815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:07.075815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:07.075815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 624/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:07.080492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f50ac-0afa-4e11-93e4-f47408c1d77f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:07.181562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f50ac-0afa-4e11-93e4-f47408c1d77f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:07.214360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ba899-dc33-430c-9f36-6a8c253fce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:07.215474 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ba899-dc33-430c-9f36-6a8c253fce51 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:07.215736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ba899-dc33-430c-9f36-6a8c253fce51 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:07.216004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ba899-dc33-430c-9f36-6a8c253fce51 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:07.216664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ba899-dc33-430c-9f36-6a8c253fce51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:07.216964 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ba899-dc33-430c-9f36-6a8c253fce51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:07.218319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ba899-dc33-430c-9f36-6a8c253fce51 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:07.232275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 628/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:07.238313 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4eb2e-0bcd-4906-8009-c5cebc59c7bf None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:07.338174 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4eb2e-0bcd-4906-8009-c5cebc59c7bf None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:07.365135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:07.372953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88a18e-d679-44d6-8058-d90f73af09bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:07.373587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88a18e-d679-44d6-8058-d90f73af09bd None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:07.373861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88a18e-d679-44d6-8058-d90f73af09bd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:07.374133 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88a18e-d679-44d6-8058-d90f73af09bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:07.381308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88a18e-d679-44d6-8058-d90f73af09bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:07.382723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88a18e-d679-44d6-8058-d90f73af09bd None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-5b88a18e-d679-44d6-8058-d90f73af09bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:23:07.393317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:07.394440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:07.395618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:07.395618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:07.395618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:23:07.395618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:23:07.395618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:23:07.395618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:23:07.395618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:23:07.395618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:23:07.395618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:23:07.395618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:07.395618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 625/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:07.400556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ba899-dc33-430c-9f36-6a8c253fce51 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:07.457035 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:07.500656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ba899-dc33-430c-9f36-6a8c253fce51 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:07.533344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134af83-41f2-4825-959b-692c62dac3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:07.533943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134af83-41f2-4825-959b-692c62dac3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:07.534222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134af83-41f2-4825-959b-692c62dac3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:07.534494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134af83-41f2-4825-959b-692c62dac3c0 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:07.535214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134af83-41f2-4825-959b-692c62dac3c0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:07.535520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134af83-41f2-4825-959b-692c62dac3c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:07.536960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134af83-41f2-4825-959b-692c62dac3c0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:07.554326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 629/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 22:23:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:07.560405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88a18e-d679-44d6-8058-d90f73af09bd None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:07.659923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88a18e-d679-44d6-8058-d90f73af09bd None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:07.692282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7ebc9-b362-425f-9995-84c240abfee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:07.692873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7ebc9-b362-425f-9995-84c240abfee3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:07.693148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7ebc9-b362-425f-9995-84c240abfee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:07.693434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7ebc9-b362-425f-9995-84c240abfee3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:07.746800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:07.759064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7ebc9-b362-425f-9995-84c240abfee3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:07.760547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7ebc9-b362-425f-9995-84c240abfee3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:07.859509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 626/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:23:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:23:07.865489 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134af83-41f2-4825-959b-692c62dac3c0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:07.964355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134af83-41f2-4825-959b-692c62dac3c0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:07.977637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:07.999803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf39674-825b-4d58-a790-efae16c49688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:08.000447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf39674-825b-4d58-a790-efae16c49688 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:08.000723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf39674-825b-4d58-a790-efae16c49688 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:08.000990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf39674-825b-4d58-a790-efae16c49688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:08.011066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf39674-825b-4d58-a790-efae16c49688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:08.014287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf39674-825b-4d58-a790-efae16c49688 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ebf39674-825b-4d58-a790-efae16c49688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:23:08.027259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:08.028519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:23:08.029810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:23:08.029810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:08.029810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:08.029810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 630/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:08.034252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7ebc9-b362-425f-9995-84c240abfee3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:08.119363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:08.131753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7ebc9-b362-425f-9995-84c240abfee3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:08.166779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bbf7c-dc95-4b82-894c-cc94bf991001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:08.167382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bbf7c-dc95-4b82-894c-cc94bf991001 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:08.167661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bbf7c-dc95-4b82-894c-cc94bf991001 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:08.167933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bbf7c-dc95-4b82-894c-cc94bf991001 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:08.168610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4bbf7c-dc95-4b82-894c-cc94bf991001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:08.168906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4bbf7c-dc95-4b82-894c-cc94bf991001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:08.170258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4bbf7c-dc95-4b82-894c-cc94bf991001 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:08.181466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 627/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:08.187762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf39674-825b-4d58-a790-efae16c49688 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:08.285579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf39674-825b-4d58-a790-efae16c49688 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:08.319111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4409383-3bbb-4d42-b640-511a1afcc1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:08.319722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4409383-3bbb-4d42-b640-511a1afcc1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:08.320001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4409383-3bbb-4d42-b640-511a1afcc1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:08.320304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4409383-3bbb-4d42-b640-511a1afcc1b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:08.327058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4409383-3bbb-4d42-b640-511a1afcc1b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:08.328566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4409383-3bbb-4d42-b640-511a1afcc1b0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-e4409383-3bbb-4d42-b640-511a1afcc1b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:23:08.339613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:08.340786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:08.342065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:23:08.342065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:23:08.342065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:23:08.342065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:23:08.342065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:23:08.342065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:23:08.342065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:23:08.342065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:08.342065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 631/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:23:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:08.346567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bbf7c-dc95-4b82-894c-cc94bf991001 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:08.444690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bbf7c-dc95-4b82-894c-cc94bf991001 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:08.476989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264995ac-d526-4aa8-bf8c-d026937fd179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:08.477580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264995ac-d526-4aa8-bf8c-d026937fd179 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:08.477855 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264995ac-d526-4aa8-bf8c-d026937fd179 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:08.478123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264995ac-d526-4aa8-bf8c-d026937fd179 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:08.478793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264995ac-d526-4aa8-bf8c-d026937fd179 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:08.479093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264995ac-d526-4aa8-bf8c-d026937fd179 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:08.480528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264995ac-d526-4aa8-bf8c-d026937fd179 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:08.492662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 628/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:23:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:08.499227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4409383-3bbb-4d42-b640-511a1afcc1b0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:08.525270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:08.598573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4409383-3bbb-4d42-b640-511a1afcc1b0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:08.633129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d23178-3260-45d3-9b46-cfdc73708f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:08.633721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d23178-3260-45d3-9b46-cfdc73708f90 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:08.634026 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d23178-3260-45d3-9b46-cfdc73708f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:08.634300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d23178-3260-45d3-9b46-cfdc73708f90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:08.635057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d23178-3260-45d3-9b46-cfdc73708f90 None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=db1d7e39e8c14f5db40efa743fc3b652) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:08.635363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d23178-3260-45d3-9b46-cfdc73708f90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:08.636880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d23178-3260-45d3-9b46-cfdc73708f90 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:08.642646 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 632/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:23:08 2022] GET /identity/v3/role_assignments?user.id=db1d7e39e8c14f5db40efa743fc3b652&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=108664305d2546949f7327b65d4cb667 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:10.618135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca09f11-e7e5-4429-a86e-27d0a00f3216 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:10.618751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca09f11-e7e5-4429-a86e-27d0a00f3216 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:10.618751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca09f11-e7e5-4429-a86e-27d0a00f3216 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:10.620135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 629/1261] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:23:10.626012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011466a5-0a78-47d8-85c6-46bdf3e9df6b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:10.626434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011466a5-0a78-47d8-85c6-46bdf3e9df6b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:10.626570 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011466a5-0a78-47d8-85c6-46bdf3e9df6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:10.644505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-011466a5-0a78-47d8-85c6-46bdf3e9df6b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:23:10.645689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-011466a5-0a78-47d8-85c6-46bdf3e9df6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:23:10.715929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 633/1262] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:23:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:23:10.738800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca09f11-e7e5-4429-a86e-27d0a00f3216 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:10.749839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:10.837408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca09f11-e7e5-4429-a86e-27d0a00f3216 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:10.871531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaae72f-da4d-483d-9aa7-a225ec1d8752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:10.872229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaae72f-da4d-483d-9aa7-a225ec1d8752 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:10.872548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaae72f-da4d-483d-9aa7-a225ec1d8752 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:10.873033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaae72f-da4d-483d-9aa7-a225ec1d8752 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:10.883480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaae72f-da4d-483d-9aa7-a225ec1d8752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-dbaae72f-da4d-483d-9aa7-a225ec1d8752 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:23:10.889065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:10.890311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:10.891221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 630/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:23:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:10.896278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011466a5-0a78-47d8-85c6-46bdf3e9df6b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:10.915775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:10.994936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011466a5-0a78-47d8-85c6-46bdf3e9df6b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:11.030047 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399d878-f405-4c74-bb2a-cdd5e6a02898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:11.031124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399d878-f405-4c74-bb2a-cdd5e6a02898 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:11.031124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399d878-f405-4c74-bb2a-cdd5e6a02898 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:11.031293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399d878-f405-4c74-bb2a-cdd5e6a02898 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:11.031832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399d878-f405-4c74-bb2a-cdd5e6a02898 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:11.032175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399d878-f405-4c74-bb2a-cdd5e6a02898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:11.033540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399d878-f405-4c74-bb2a-cdd5e6a02898 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:11.041814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 634/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:23:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:11.048294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaae72f-da4d-483d-9aa7-a225ec1d8752 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:11.114894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:11.147488 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaae72f-da4d-483d-9aa7-a225ec1d8752 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:11.180477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014b2e7-8ebd-403f-bc05-fd5ad962fb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:11.181094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014b2e7-8ebd-403f-bc05-fd5ad962fb10 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:11.181366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014b2e7-8ebd-403f-bc05-fd5ad962fb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:11.181633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014b2e7-8ebd-403f-bc05-fd5ad962fb10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:11.192577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014b2e7-8ebd-403f-bc05-fd5ad962fb10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:11.194062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014b2e7-8ebd-403f-bc05-fd5ad962fb10 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-0014b2e7-8ebd-403f-bc05-fd5ad962fb10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:23:11.208328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:11.209455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:23:11.210520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:23:11.210520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:11.210520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:11.210520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 631/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:11.215292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399d878-f405-4c74-bb2a-cdd5e6a02898 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:11.297317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:11.316664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399d878-f405-4c74-bb2a-cdd5e6a02898 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:11.349894 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bdd45-b52f-4f0d-98bd-38c1f547dc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:11.350793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bdd45-b52f-4f0d-98bd-38c1f547dc7d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:11.350793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bdd45-b52f-4f0d-98bd-38c1f547dc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:11.351072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bdd45-b52f-4f0d-98bd-38c1f547dc7d None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:11.351714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bdd45-b52f-4f0d-98bd-38c1f547dc7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:11.352050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bdd45-b52f-4f0d-98bd-38c1f547dc7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:11.353425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bdd45-b52f-4f0d-98bd-38c1f547dc7d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:11.365116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 635/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:11.370837 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014b2e7-8ebd-403f-bc05-fd5ad962fb10 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:11.471412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014b2e7-8ebd-403f-bc05-fd5ad962fb10 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:11.491562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:11.505900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9422d-7e76-4ae6-a0c9-913fc472ed47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:11.506519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9422d-7e76-4ae6-a0c9-913fc472ed47 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:11.506795 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9422d-7e76-4ae6-a0c9-913fc472ed47 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:11.507072 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9422d-7e76-4ae6-a0c9-913fc472ed47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:11.514371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9422d-7e76-4ae6-a0c9-913fc472ed47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:11.515805 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9422d-7e76-4ae6-a0c9-913fc472ed47 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-11a9422d-7e76-4ae6-a0c9-913fc472ed47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:23:11.526419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:11.527703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:11.528878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:11.528878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:11.528878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:23:11.528878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:23:11.528878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:23:11.528878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:23:11.528878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:23:11.528878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:23:11.528878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:23:11.528878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:11.528878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 632/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:11.533708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bdd45-b52f-4f0d-98bd-38c1f547dc7d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:11.633977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bdd45-b52f-4f0d-98bd-38c1f547dc7d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:11.668036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b050da-d539-4d1e-90ec-2620d3ce7e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:11.668654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b050da-d539-4d1e-90ec-2620d3ce7e04 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:11.668932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b050da-d539-4d1e-90ec-2620d3ce7e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:11.669204 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b050da-d539-4d1e-90ec-2620d3ce7e04 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:11.669921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b050da-d539-4d1e-90ec-2620d3ce7e04 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:11.670342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b050da-d539-4d1e-90ec-2620d3ce7e04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:11.671756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b050da-d539-4d1e-90ec-2620d3ce7e04 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:11.680786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:11.689264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 636/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 22:23:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:11.694895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9422d-7e76-4ae6-a0c9-913fc472ed47 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:11.793756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9422d-7e76-4ae6-a0c9-913fc472ed47 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:11.828026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e174f-dd88-4a3c-898e-999415d787ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:11.828647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e174f-dd88-4a3c-898e-999415d787ed None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:11.828916 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e174f-dd88-4a3c-898e-999415d787ed None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:11.829195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e174f-dd88-4a3c-898e-999415d787ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:11.841531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e174f-dd88-4a3c-898e-999415d787ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:11.842950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e174f-dd88-4a3c-898e-999415d787ed None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-330e174f-dd88-4a3c-898e-999415d787ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:23:11.856936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:11.858190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:23:11.859291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:23:11.859291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:11.859291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:11.859291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 633/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:11.864000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b050da-d539-4d1e-90ec-2620d3ce7e04 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:11.964089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b050da-d539-4d1e-90ec-2620d3ce7e04 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:11.999315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1929f-28a1-4548-b674-3f39c5e5a739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:11.999912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1929f-28a1-4548-b674-3f39c5e5a739 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:12.000221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1929f-28a1-4548-b674-3f39c5e5a739 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:12.000521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1929f-28a1-4548-b674-3f39c5e5a739 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:12.001152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1929f-28a1-4548-b674-3f39c5e5a739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:12.001452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1929f-28a1-4548-b674-3f39c5e5a739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:12.002800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1929f-28a1-4548-b674-3f39c5e5a739 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:12.013671 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 637/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:12.019393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e174f-dd88-4a3c-898e-999415d787ed None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:12.050304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:12.115365 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e174f-dd88-4a3c-898e-999415d787ed None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:12.150854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0c7fb-364f-4c4d-80e9-b5159bca156d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:12.151471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c7fb-364f-4c4d-80e9-b5159bca156d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:12.151748 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c7fb-364f-4c4d-80e9-b5159bca156d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:12.152068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c7fb-364f-4c4d-80e9-b5159bca156d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:12.158728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0c7fb-364f-4c4d-80e9-b5159bca156d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:12.160175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0c7fb-364f-4c4d-80e9-b5159bca156d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-04f0c7fb-364f-4c4d-80e9-b5159bca156d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:23:12.171388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:12.172573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:12.173465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:23:12.173465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:23:12.173465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:23:12.173465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:23:12.173465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:23:12.173465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:23:12.173465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:23:12.173465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:12.173465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 634/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:23:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:12.178411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1929f-28a1-4548-b674-3f39c5e5a739 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:12.236523 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:12.278576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1929f-28a1-4548-b674-3f39c5e5a739 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:12.312648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab41f6-9fb8-4273-8e8e-702dc21fb831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:12.313157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab41f6-9fb8-4273-8e8e-702dc21fb831 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:12.313435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab41f6-9fb8-4273-8e8e-702dc21fb831 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:12.313705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab41f6-9fb8-4273-8e8e-702dc21fb831 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:12.314392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab41f6-9fb8-4273-8e8e-702dc21fb831 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:12.314703 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab41f6-9fb8-4273-8e8e-702dc21fb831 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:12.316147 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab41f6-9fb8-4273-8e8e-702dc21fb831 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:12.327893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 638/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:23:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:12.334218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0c7fb-364f-4c4d-80e9-b5159bca156d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:12.424706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:12.433088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0c7fb-364f-4c4d-80e9-b5159bca156d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:12.466078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac63a8-7713-4d48-b986-a979ffc9d682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:12.466709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac63a8-7713-4d48-b986-a979ffc9d682 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:12.466985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac63a8-7713-4d48-b986-a979ffc9d682 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:12.467256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac63a8-7713-4d48-b986-a979ffc9d682 None admin] PATH_INFO: `/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/db1d7e39e8c14f5db40efa743fc3b652/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:12.469126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac63a8-7713-4d48-b986-a979ffc9d682 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d6a8b879d4edeaaa815c4556ba6a4, user_id=db1d7e39e8c14f5db40efa743fc3b652, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:12.470575 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac63a8-7713-4d48-b986-a979ffc9d682 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:12.613152 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 635/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 22:23:12 2022] PUT /identity/v3/projects/9a9d6a8b879d4edeaaa815c4556ba6a4/users/db1d7e39e8c14f5db40efa743fc3b652/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:23:14.629953 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0954b-d009-4255-8db3-ea6af390ea83 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:14.630509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0954b-d009-4255-8db3-ea6af390ea83 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:14.630509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0954b-d009-4255-8db3-ea6af390ea83 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:14.631845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 639/1274] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:23:14.637878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fa3f1-8a20-4ef5-bb03-82ccd4b494cf None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:14.638320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fa3f1-8a20-4ef5-bb03-82ccd4b494cf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:14.638320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fa3f1-8a20-4ef5-bb03-82ccd4b494cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:14.657296 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ee6fa3f1-8a20-4ef5-bb03-82ccd4b494cf None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:23:14.658497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ee6fa3f1-8a20-4ef5-bb03-82ccd4b494cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:23:14.724325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 636/1275] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:23:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:23:14.748483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0954b-d009-4255-8db3-ea6af390ea83 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:14.849551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0954b-d009-4255-8db3-ea6af390ea83 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:14.866843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:14.884001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b8c03-6a62-40c8-b844-074c0364ce54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:14.884603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b8c03-6a62-40c8-b844-074c0364ce54 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:14.884879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b8c03-6a62-40c8-b844-074c0364ce54 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:14.885156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b8c03-6a62-40c8-b844-074c0364ce54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:14.895907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b8c03-6a62-40c8-b844-074c0364ce54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c76b8c03-6a62-40c8-b844-074c0364ce54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 22:23:14.900917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:14.902006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:14.903183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 640/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 22:23:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 22:23:14.907997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fa3f1-8a20-4ef5-bb03-82ccd4b494cf None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:14.987832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:15.006383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fa3f1-8a20-4ef5-bb03-82ccd4b494cf None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:15.039291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919f473-37dd-4f10-837a-120e387af374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:15.039881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f473-37dd-4f10-837a-120e387af374 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:15.040266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f473-37dd-4f10-837a-120e387af374 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:15.040546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f473-37dd-4f10-837a-120e387af374 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:15.041209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919f473-37dd-4f10-837a-120e387af374 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:15.041497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919f473-37dd-4f10-837a-120e387af374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:15.043653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919f473-37dd-4f10-837a-120e387af374 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:15.051509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 637/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 22:23:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:15.057249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b8c03-6a62-40c8-b844-074c0364ce54 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:15.160771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b8c03-6a62-40c8-b844-074c0364ce54 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:15.195173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7100e-a5ae-469a-941c-ebc05cbec443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:15.195795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7100e-a5ae-469a-941c-ebc05cbec443 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:15.196146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7100e-a5ae-469a-941c-ebc05cbec443 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:15.196410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7100e-a5ae-469a-941c-ebc05cbec443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:15.206643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7100e-a5ae-469a-941c-ebc05cbec443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:15.208127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7100e-a5ae-469a-941c-ebc05cbec443 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-2ac7100e-a5ae-469a-941c-ebc05cbec443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:23:15.222619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:15.223776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:23:15.237207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:23:15.237207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:15.237207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:15.237207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 641/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:15.237513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919f473-37dd-4f10-837a-120e387af374 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:15.329410 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919f473-37dd-4f10-837a-120e387af374 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:15.363121 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2f963-f490-40bb-be21-fe1596e0dfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:15.363718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2f963-f490-40bb-be21-fe1596e0dfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:15.364127 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2f963-f490-40bb-be21-fe1596e0dfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:15.364428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2f963-f490-40bb-be21-fe1596e0dfa3 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:15.365065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2f963-f490-40bb-be21-fe1596e0dfa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:15.365373 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2f963-f490-40bb-be21-fe1596e0dfa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:15.366726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2f963-f490-40bb-be21-fe1596e0dfa3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:15.379169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 638/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:15.384675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7100e-a5ae-469a-941c-ebc05cbec443 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:15.415775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:15.482925 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7100e-a5ae-469a-941c-ebc05cbec443 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:15.518585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c31432-742a-4724-b27f-a1eeeaceff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:15.519319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c31432-742a-4724-b27f-a1eeeaceff24 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:15.519593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c31432-742a-4724-b27f-a1eeeaceff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:15.519860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c31432-742a-4724-b27f-a1eeeaceff24 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:15.526715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c31432-742a-4724-b27f-a1eeeaceff24 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:15.528362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c31432-742a-4724-b27f-a1eeeaceff24 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-07c31432-742a-4724-b27f-a1eeeaceff24 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 22:23:15.539575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:15.540706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:15.541708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:15.541708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:15.541708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 22:23:15.541708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 22:23:15.541708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 22:23:15.541708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 22:23:15.541708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 22:23:15.541708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 22:23:15.541708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 22:23:15.541708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:15.541708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 642/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:15.546732 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2f963-f490-40bb-be21-fe1596e0dfa3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:15.550636 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:15.644102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2f963-f490-40bb-be21-fe1596e0dfa3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:15.676975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4ef9d-dd06-4cb0-a570-4e8eda8f888c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:15.677506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4ef9d-dd06-4cb0-a570-4e8eda8f888c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:15.677783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4ef9d-dd06-4cb0-a570-4e8eda8f888c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:15.678055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4ef9d-dd06-4cb0-a570-4e8eda8f888c None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:15.678771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4ef9d-dd06-4cb0-a570-4e8eda8f888c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:15.679074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4ef9d-dd06-4cb0-a570-4e8eda8f888c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:15.680553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4ef9d-dd06-4cb0-a570-4e8eda8f888c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:15.697680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 639/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 22:23:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:15.704289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c31432-742a-4724-b27f-a1eeeaceff24 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:15.794381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:15.803313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c31432-742a-4724-b27f-a1eeeaceff24 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:15.836318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32835eb5-5c47-4f9b-8267-900732b21014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:15.836934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835eb5-5c47-4f9b-8267-900732b21014 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:15.837206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835eb5-5c47-4f9b-8267-900732b21014 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:15.837473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835eb5-5c47-4f9b-8267-900732b21014 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:15.904543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32835eb5-5c47-4f9b-8267-900732b21014 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:15.906084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32835eb5-5c47-4f9b-8267-900732b21014 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:15.994858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:16.005684 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 643/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 22:23:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 22:23:16.011563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4ef9d-dd06-4cb0-a570-4e8eda8f888c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:16.072775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:16.111569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4ef9d-dd06-4cb0-a570-4e8eda8f888c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:16.144359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f505e9-6341-4042-ba7c-a13dcb8a35b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:16.145018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f505e9-6341-4042-ba7c-a13dcb8a35b6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:16.145291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f505e9-6341-4042-ba7c-a13dcb8a35b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:16.145562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f505e9-6341-4042-ba7c-a13dcb8a35b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:16.156932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f505e9-6341-4042-ba7c-a13dcb8a35b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:16.158360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f505e9-6341-4042-ba7c-a13dcb8a35b6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-b4f505e9-6341-4042-ba7c-a13dcb8a35b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 22:23:16.172987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:16.174130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 22:23:16.175163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 22:23:16.175163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 22:23:16.175163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:16.175163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 640/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:16.179923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32835eb5-5c47-4f9b-8267-900732b21014 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:16.278199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32835eb5-5c47-4f9b-8267-900732b21014 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:16.312546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74873967-8cf9-4768-bfad-9c2e80ce663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:16.313175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74873967-8cf9-4768-bfad-9c2e80ce663f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:16.313451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74873967-8cf9-4768-bfad-9c2e80ce663f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:16.313721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74873967-8cf9-4768-bfad-9c2e80ce663f None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:16.314360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74873967-8cf9-4768-bfad-9c2e80ce663f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:16.314666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74873967-8cf9-4768-bfad-9c2e80ce663f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:16.316077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74873967-8cf9-4768-bfad-9c2e80ce663f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:16.327534 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 644/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 22:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:16.333164 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f505e9-6341-4042-ba7c-a13dcb8a35b6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:16.431796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f505e9-6341-4042-ba7c-a13dcb8a35b6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:16.445011 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:16.465765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202327ff-4484-457a-a0c8-1114ebba8881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:16.466380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202327ff-4484-457a-a0c8-1114ebba8881 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:16.466653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202327ff-4484-457a-a0c8-1114ebba8881 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:16.466921 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202327ff-4484-457a-a0c8-1114ebba8881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:16.473341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202327ff-4484-457a-a0c8-1114ebba8881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:16.474749 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202327ff-4484-457a-a0c8-1114ebba8881 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-202327ff-4484-457a-a0c8-1114ebba8881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 22:23:16.485541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:16.486565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:16.487762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 22:23:16.487762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 22:23:16.487762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 22:23:16.487762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 22:23:16.487762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 22:23:16.487762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 22:23:16.487762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 22:23:16.487762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Aug 07 22:23:16.487762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 641/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 22:23:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:16.492790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74873967-8cf9-4768-bfad-9c2e80ce663f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:16.544141 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:16.592327 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74873967-8cf9-4768-bfad-9c2e80ce663f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:16.624932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482bc16-c7ad-48c5-bea0-f9efae16c28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:16.625559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482bc16-c7ad-48c5-bea0-f9efae16c28c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:16.625868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482bc16-c7ad-48c5-bea0-f9efae16c28c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:16.626139 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482bc16-c7ad-48c5-bea0-f9efae16c28c None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:16.626824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482bc16-c7ad-48c5-bea0-f9efae16c28c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:16.627125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482bc16-c7ad-48c5-bea0-f9efae16c28c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:16.628624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482bc16-c7ad-48c5-bea0-f9efae16c28c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:16.641675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 645/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 22:23:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:16.647729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202327ff-4484-457a-a0c8-1114ebba8881 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:16.745589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202327ff-4484-457a-a0c8-1114ebba8881 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:16.779299 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89da38-ae7a-49e4-9e6e-fedf55fc11f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:16.779884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89da38-ae7a-49e4-9e6e-fedf55fc11f5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:16.780191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89da38-ae7a-49e4-9e6e-fedf55fc11f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:16.780470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89da38-ae7a-49e4-9e6e-fedf55fc11f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:16.781187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89da38-ae7a-49e4-9e6e-fedf55fc11f5 None admin] RBAC: Adding query filter params (role.id=108664305d2546949f7327b65d4cb667, scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4, user.id=db1d7e39e8c14f5db40efa743fc3b652) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:16.781494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89da38-ae7a-49e4-9e6e-fedf55fc11f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:16.782900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89da38-ae7a-49e4-9e6e-fedf55fc11f5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:16.788555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 642/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 22:23:16 2022] GET /identity/v3/role_assignments?user.id=db1d7e39e8c14f5db40efa743fc3b652&scope.project.id=9a9d6a8b879d4edeaaa815c4556ba6a4&role.id=108664305d2546949f7327b65d4cb667 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:18.801618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a3e09-688f-4aa8-9444-c544d60fed48 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:18.802143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a3e09-688f-4aa8-9444-c544d60fed48 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:18.802264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a3e09-688f-4aa8-9444-c544d60fed48 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:18.803510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 646/1288] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:23:18.812428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef64d7d-fbf8-4fa1-a7ee-1560e5fbc4ea None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:18.812428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef64d7d-fbf8-4fa1-a7ee-1560e5fbc4ea None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:18.812428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef64d7d-fbf8-4fa1-a7ee-1560e5fbc4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:18.828156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-fef64d7d-fbf8-4fa1-a7ee-1560e5fbc4ea None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:23:18.829348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-fef64d7d-fbf8-4fa1-a7ee-1560e5fbc4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:23:18.857609 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:18.893728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 643/1289] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:23:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:23:18.915676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a3e09-688f-4aa8-9444-c544d60fed48 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:19.018161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a3e09-688f-4aa8-9444-c544d60fed48 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:19.028606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:19.050716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a539a-a1a6-4903-b938-4f8ccb941d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:19.051333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a539a-a1a6-4903-b938-4f8ccb941d26 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:19.051612 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a539a-a1a6-4903-b938-4f8ccb941d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:19.051930 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a539a-a1a6-4903-b938-4f8ccb941d26 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:19.052602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a539a-a1a6-4903-b938-4f8ccb941d26 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:19.053957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a539a-a1a6-4903-b938-4f8ccb941d26 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-e45a539a-a1a6-4903-b938-4f8ccb941d26 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:19.062896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:19.067598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:19.068711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 647/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 22:23:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:19.070173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef64d7d-fbf8-4fa1-a7ee-1560e5fbc4ea None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:19.170092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef64d7d-fbf8-4fa1-a7ee-1560e5fbc4ea None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:19.207990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a7e1f-9f65-4427-b84e-4e1d567eb3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:19.208651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a7e1f-9f65-4427-b84e-4e1d567eb3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:19.208932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a7e1f-9f65-4427-b84e-4e1d567eb3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:19.209205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a7e1f-9f65-4427-b84e-4e1d567eb3e5 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:19.209828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a7e1f-9f65-4427-b84e-4e1d567eb3e5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:19.210134 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a7e1f-9f65-4427-b84e-4e1d567eb3e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:19.211494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a7e1f-9f65-4427-b84e-4e1d567eb3e5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:19.216219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 644/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Sun Aug 7 22:23:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:19.222033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a539a-a1a6-4903-b938-4f8ccb941d26 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:19.323656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a539a-a1a6-4903-b938-4f8ccb941d26 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:19.356543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d323a-3bf8-49b3-8a4f-84e9d0b440fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:19.357134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d323a-3bf8-49b3-8a4f-84e9d0b440fa None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:19.357410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d323a-3bf8-49b3-8a4f-84e9d0b440fa None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:19.357687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d323a-3bf8-49b3-8a4f-84e9d0b440fa None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:19.358303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d323a-3bf8-49b3-8a4f-84e9d0b440fa None admin] RBAC: Adding query filter params (type=placement) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:19.358608 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d323a-3bf8-49b3-8a4f-84e9d0b440fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:19.359969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d323a-3bf8-49b3-8a4f-84e9d0b440fa None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:19.364381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 648/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Sun Aug 7 22:23:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:21.346940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ec33-4de3-4f0a-a64d-e577d5bd09b2 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:21.347496 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ec33-4de3-4f0a-a64d-e577d5bd09b2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:21.347496 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ec33-4de3-4f0a-a64d-e577d5bd09b2 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:21.348878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 645/1293] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:23:21.354159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91c580-d96e-47e5-884b-b1be3fa31651 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:21.354581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91c580-d96e-47e5-884b-b1be3fa31651 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:21.354711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91c580-d96e-47e5-884b-b1be3fa31651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:21.373751 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cf91c580-d96e-47e5-884b-b1be3fa31651 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:23:21.374965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cf91c580-d96e-47e5-884b-b1be3fa31651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:23:21.404982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:21.441661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 649/1294] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:23:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:23:21.463863 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ec33-4de3-4f0a-a64d-e577d5bd09b2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:21.494990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:21.563934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ec33-4de3-4f0a-a64d-e577d5bd09b2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:21.597167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe274402-fe78-4575-9932-5995b4f62b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:21.597696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274402-fe78-4575-9932-5995b4f62b21 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:21.597981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274402-fe78-4575-9932-5995b4f62b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:21.598249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274402-fe78-4575-9932-5995b4f62b21 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:21.598888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe274402-fe78-4575-9932-5995b4f62b21 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:21.600266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe274402-fe78-4575-9932-5995b4f62b21 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:21.677350 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 646/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 22:23:21 2022] POST /identity/v3/services => generated 246 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:23:23.688445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f6e2d-617b-4cf0-8f1e-a3912b3f8852 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:23.689006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f6e2d-617b-4cf0-8f1e-a3912b3f8852 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:23.689006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f6e2d-617b-4cf0-8f1e-a3912b3f8852 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:23.690352 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 650/1296] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:23:23.695705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd247c-5a6f-4e14-b67e-286c1d7ba5ca None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:23.696244 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd247c-5a6f-4e14-b67e-286c1d7ba5ca None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:23.696431 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd247c-5a6f-4e14-b67e-286c1d7ba5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:23.715989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4ccd247c-5a6f-4e14-b67e-286c1d7ba5ca None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:23:23.717278 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4ccd247c-5a6f-4e14-b67e-286c1d7ba5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:23:23.788789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 647/1297] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:23:23 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:23:23.810791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f6e2d-617b-4cf0-8f1e-a3912b3f8852 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:23.911186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f6e2d-617b-4cf0-8f1e-a3912b3f8852 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:23.945171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6d8d2-2704-45c6-a1aa-7c0f2be0e0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:23.945861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6d8d2-2704-45c6-a1aa-7c0f2be0e0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:23.946143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6d8d2-2704-45c6-a1aa-7c0f2be0e0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:23.946418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6d8d2-2704-45c6-a1aa-7c0f2be0e0d1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:23.947006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6d8d2-2704-45c6-a1aa-7c0f2be0e0d1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:23.948402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6d8d2-2704-45c6-a1aa-7c0f2be0e0d1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c1a6d8d2-2704-45c6-a1aa-7c0f2be0e0d1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:23.957479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:23.958674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:23.959583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 651/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 22:23:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:23.965351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd247c-5a6f-4e14-b67e-286c1d7ba5ca None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:23.983667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:24.063122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd247c-5a6f-4e14-b67e-286c1d7ba5ca None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:24.096785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc824725-f601-450d-a378-3916ed4c1906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:24.097383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc824725-f601-450d-a378-3916ed4c1906 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:24.097658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc824725-f601-450d-a378-3916ed4c1906 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:24.097954 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc824725-f601-450d-a378-3916ed4c1906 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:24.098569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc824725-f601-450d-a378-3916ed4c1906 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:24.098867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc824725-f601-450d-a378-3916ed4c1906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:24.100234 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc824725-f601-450d-a378-3916ed4c1906 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:24.105211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 648/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Sun Aug 7 22:23:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:24.111349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6d8d2-2704-45c6-a1aa-7c0f2be0e0d1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:24.143319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:24.214978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6d8d2-2704-45c6-a1aa-7c0f2be0e0d1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:24.250009 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e08c1-116e-43fb-ae3e-5f56e0dbc73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:24.250614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e08c1-116e-43fb-ae3e-5f56e0dbc73a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:24.250889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e08c1-116e-43fb-ae3e-5f56e0dbc73a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:24.251160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e08c1-116e-43fb-ae3e-5f56e0dbc73a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:24.251858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e08c1-116e-43fb-ae3e-5f56e0dbc73a None admin] RBAC: Adding query filter params (interface=public, service_id=ed2b9cc6e54d4f6883e991e7525e539e, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:24.252192 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e08c1-116e-43fb-ae3e-5f56e0dbc73a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:24.253606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e08c1-116e-43fb-ae3e-5f56e0dbc73a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:24.258656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 652/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 22:23:24 2022] GET /identity/v3/endpoints?service_id=ed2b9cc6e54d4f6883e991e7525e539e&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:24.265018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc824725-f601-450d-a378-3916ed4c1906 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:24.348062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:24.366985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc824725-f601-450d-a378-3916ed4c1906 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:24.402997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadef62-8de3-4b79-bec4-9b68e2731348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:24.403593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadef62-8de3-4b79-bec4-9b68e2731348 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:24.403878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadef62-8de3-4b79-bec4-9b68e2731348 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:24.404186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadef62-8de3-4b79-bec4-9b68e2731348 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:24.404792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecadef62-8de3-4b79-bec4-9b68e2731348 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:24.406140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecadef62-8de3-4b79-bec4-9b68e2731348 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:24.412938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 649/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 22:23:24 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:23:26.403269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd5a8b-ec7c-49b8-b7ba-e676db089698 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:26.403847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd5a8b-ec7c-49b8-b7ba-e676db089698 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:26.403847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd5a8b-ec7c-49b8-b7ba-e676db089698 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:26.405262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 653/1302] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:23:26.410413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd118a-f4eb-4954-bbc8-079070e4b299 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:26.410826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd118a-f4eb-4954-bbc8-079070e4b299 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:26.410958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd118a-f4eb-4954-bbc8-079070e4b299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:26.429754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f1dd118a-f4eb-4954-bbc8-079070e4b299 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:23:26.431049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f1dd118a-f4eb-4954-bbc8-079070e4b299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:23:26.496199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 650/1303] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:23:26 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 22:23:26.518674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd5a8b-ec7c-49b8-b7ba-e676db089698 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:26.612235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:26.618804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd5a8b-ec7c-49b8-b7ba-e676db089698 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:26.651618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e078a7e-85ca-4704-952d-6c5f138f9ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:26.652273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e078a7e-85ca-4704-952d-6c5f138f9ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:26.652552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e078a7e-85ca-4704-952d-6c5f138f9ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:26.652824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e078a7e-85ca-4704-952d-6c5f138f9ff3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:26.653456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e078a7e-85ca-4704-952d-6c5f138f9ff3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:26.654782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e078a7e-85ca-4704-952d-6c5f138f9ff3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-3e078a7e-85ca-4704-952d-6c5f138f9ff3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 22:23:26.664279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 22:23:26.665504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Aug 07 22:23:26.666638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 654/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 22:23:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 22:23:26.672119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd118a-f4eb-4954-bbc8-079070e4b299 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:26.746471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:26.768951 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd118a-f4eb-4954-bbc8-079070e4b299 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:26.801655 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897dce0b-1cf5-4b13-939f-695f17e0b967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:26.802249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dce0b-1cf5-4b13-939f-695f17e0b967 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:26.802525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dce0b-1cf5-4b13-939f-695f17e0b967 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:26.802798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dce0b-1cf5-4b13-939f-695f17e0b967 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:26.803422 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897dce0b-1cf5-4b13-939f-695f17e0b967 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:23:26.803727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897dce0b-1cf5-4b13-939f-695f17e0b967 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:26.805120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897dce0b-1cf5-4b13-939f-695f17e0b967 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:26.810482 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 651/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Sun Aug 7 22:23:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:23:26.816356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e078a7e-85ca-4704-952d-6c5f138f9ff3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:23:26.914238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e078a7e-85ca-4704-952d-6c5f138f9ff3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:23:26.946377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104a22a-b878-464a-944c-b84a40eebbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:23:26.946976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104a22a-b878-464a-944c-b84a40eebbf2 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:23:26.947250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104a22a-b878-464a-944c-b84a40eebbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:23:26.947518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104a22a-b878-464a-944c-b84a40eebbf2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:23:26.948203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104a22a-b878-464a-944c-b84a40eebbf2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:23:26.950467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104a22a-b878-464a-944c-b84a40eebbf2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:23:27.026295 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 655/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:23:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:23:36.871679 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=30 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:23:37.021602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:26.873011 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937bbb1-b032-4c5d-ac30-77db6fd73bb6 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:26.874162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937bbb1-b032-4c5d-ac30-77db6fd73bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:26.874688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937bbb1-b032-4c5d-ac30-77db6fd73bb6 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:26.881797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6f771-3c2d-4028-afb3-d82a2e7bf9f7 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:26.882667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6f771-3c2d-4028-afb3-d82a2e7bf9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:26.883163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6f771-3c2d-4028-afb3-d82a2e7bf9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:26.886220 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 652/1307] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:29:26 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:26.962664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2fe6f771-3c2d-4028-afb3-d82a2e7bf9f7 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:26.964313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2fe6f771-3c2d-4028-afb3-d82a2e7bf9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:27.141577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 656/1308] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:29:27.201215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b12df-3cde-415f-9278-2cd8bbfec28d None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:27.201215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b12df-3cde-415f-9278-2cd8bbfec28d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:27.201215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b12df-3cde-415f-9278-2cd8bbfec28d None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:27.203038 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 653/1309] 10.222.0.30 () {54 vars in 965 bytes} [Sun Aug 7 22:29:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:27.215477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8f0a4-d945-4e38-b6b3-6d7a6cb8f188 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:27.215766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8f0a4-d945-4e38-b6b3-6d7a6cb8f188 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:27.216063 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8f0a4-d945-4e38-b6b3-6d7a6cb8f188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:27.375051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c5c8f0a4-d945-4e38-b6b3-6d7a6cb8f188 None None] MFA Rules not processed for user `baffd4f6a8ad4a6e91d8d9c61cc1dce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:27.377150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c5c8f0a4-d945-4e38-b6b3-6d7a6cb8f188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:27.409342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:27.479829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 657/1310] 10.222.0.30 () {58 vars in 1062 bytes} [Sun Aug 7 22:29:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:27.489580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b12df-3cde-415f-9278-2cd8bbfec28d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:29:27.491867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-113b12df-3cde-415f-9278-2cd8bbfec28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:27.621738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b12df-3cde-415f-9278-2cd8bbfec28d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:29:27.702960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8dbff-634f-4614-a701-d0e1ed0aa633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:29:27.703930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8dbff-634f-4614-a701-d0e1ed0aa633 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:27.704444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8dbff-634f-4614-a701-d0e1ed0aa633 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:27.704941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8dbff-634f-4614-a701-d0e1ed0aa633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:27.706924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-31b8dbff-634f-4614-a701-d0e1ed0aa633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:27.763408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:27.820196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8dbff-634f-4614-a701-d0e1ed0aa633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:29:27.822031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8dbff-634f-4614-a701-d0e1ed0aa633 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:29:27.853253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 654/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:29:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:29:30.217058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887844a-1fd1-4970-a35c-3f8661bbbfa2 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:30.217604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887844a-1fd1-4970-a35c-3f8661bbbfa2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:30.217604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887844a-1fd1-4970-a35c-3f8661bbbfa2 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:30.218868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 658/1312] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:30.224145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cab378-51e4-48c9-9c6e-3e22290d7dc2 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:30.224573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cab378-51e4-48c9-9c6e-3e22290d7dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:30.224851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cab378-51e4-48c9-9c6e-3e22290d7dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:30.307028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-02cab378-51e4-48c9-9c6e-3e22290d7dc2 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:30.308309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-02cab378-51e4-48c9-9c6e-3e22290d7dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:30.407409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 655/1313] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:30.447022 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887844a-1fd1-4970-a35c-3f8661bbbfa2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:29:30.448597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1887844a-1fd1-4970-a35c-3f8661bbbfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:30.560296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887844a-1fd1-4970-a35c-3f8661bbbfa2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:29:30.638455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:30.642031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0e60c-3c7b-4862-8b52-e10dff141b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:29:30.642736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0e60c-3c7b-4862-8b52-e10dff141b0e service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:30.643072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0e60c-3c7b-4862-8b52-e10dff141b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:30.643295 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0e60c-3c7b-4862-8b52-e10dff141b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:30.645010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-36b0e60c-3c7b-4862-8b52-e10dff141b0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:30.759328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0e60c-3c7b-4862-8b52-e10dff141b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:29:30.760811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0e60c-3c7b-4862-8b52-e10dff141b0e service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:29:30.779664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 659/1314] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:29:33.242729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3573972-68a9-4cd8-8490-22265b121416 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:33.243245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3573972-68a9-4cd8-8490-22265b121416 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:33.243245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3573972-68a9-4cd8-8490-22265b121416 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:33.244604 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 656/1315] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:33.249809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb84efc-1c4a-44e0-8221-c84e7e744e80 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:33.250251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb84efc-1c4a-44e0-8221-c84e7e744e80 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:33.250368 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb84efc-1c4a-44e0-8221-c84e7e744e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:33.316653 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2eb84efc-1c4a-44e0-8221-c84e7e744e80 None None] MFA Rules not processed for user `3d8ca4abbdac4f939233a06b4533d187`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:33.317878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2eb84efc-1c4a-44e0-8221-c84e7e744e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:33.371819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:33.415920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 660/1316] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:29:33 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:33.455358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3573972-68a9-4cd8-8490-22265b121416 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:29:33.456934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b3573972-68a9-4cd8-8490-22265b121416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:33.466586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:33.684102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3573972-68a9-4cd8-8490-22265b121416 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:29:33.757295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a43088-2d0e-4c11-803b-d6e9a4702115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:29:33.757889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a43088-2d0e-4c11-803b-d6e9a4702115 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:33.758174 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a43088-2d0e-4c11-803b-d6e9a4702115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:33.758447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a43088-2d0e-4c11-803b-d6e9a4702115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:33.760065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b1a43088-2d0e-4c11-803b-d6e9a4702115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:33.821639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:33.869174 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a43088-2d0e-4c11-803b-d6e9a4702115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:29:33.870594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a43088-2d0e-4c11-803b-d6e9a4702115 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:29:33.888476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 657/1317] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:29:33 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:29:36.681744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b5d0f-7dd9-47a1-bb73-c9e70f85848d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:36.682321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b5d0f-7dd9-47a1-bb73-c9e70f85848d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:36.682321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b5d0f-7dd9-47a1-bb73-c9e70f85848d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:36.683644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 661/1318] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:36.689741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ea205-49ff-4899-9a1d-0129fd6c8eae None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:36.690188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ea205-49ff-4899-9a1d-0129fd6c8eae None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:36.690320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ea205-49ff-4899-9a1d-0129fd6c8eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:36.755775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ff9ea205-49ff-4899-9a1d-0129fd6c8eae None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:36.757050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ff9ea205-49ff-4899-9a1d-0129fd6c8eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:36.856157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 658/1319] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:29:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:36.904564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b03018-fd18-4459-a8ef-aecce80bfb15 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:36.905369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b03018-fd18-4459-a8ef-aecce80bfb15 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:36.905643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b03018-fd18-4459-a8ef-aecce80bfb15 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:36.906984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 662/1320] 10.222.0.30 () {54 vars in 965 bytes} [Sun Aug 7 22:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:36.912679 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbab28-c32b-4f5f-9bbf-43f50c17e12c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:36.912974 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbab28-c32b-4f5f-9bbf-43f50c17e12c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:36.913251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbab28-c32b-4f5f-9bbf-43f50c17e12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:36.940622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:36.990084 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-21cbab28-c32b-4f5f-9bbf-43f50c17e12c None None] MFA Rules not processed for user `baffd4f6a8ad4a6e91d8d9c61cc1dce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:36.991978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-21cbab28-c32b-4f5f-9bbf-43f50c17e12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:37.066829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 659/1321] 10.222.0.30 () {58 vars in 1062 bytes} [Sun Aug 7 22:29:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:37.075665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b03018-fd18-4459-a8ef-aecce80bfb15 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:29:37.077230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a6b03018-fd18-4459-a8ef-aecce80bfb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:37.179489 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b03018-fd18-4459-a8ef-aecce80bfb15 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:29:37.251209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8cccc-a7a7-4166-b554-b1d6b7af5d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:29:37.251738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8cccc-a7a7-4166-b554-b1d6b7af5d58 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:37.252105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8cccc-a7a7-4166-b554-b1d6b7af5d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:37.252331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8cccc-a7a7-4166-b554-b1d6b7af5d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:37.253921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b5c8cccc-a7a7-4166-b554-b1d6b7af5d58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:37.261240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:37.361952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8cccc-a7a7-4166-b554-b1d6b7af5d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:29:37.363396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8cccc-a7a7-4166-b554-b1d6b7af5d58 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:29:37.381694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 663/1322] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:29:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:29:40.907433 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c67e9-a9e7-490b-b1c6-f4662daad851 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:40.907988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c67e9-a9e7-490b-b1c6-f4662daad851 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:40.907988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c67e9-a9e7-490b-b1c6-f4662daad851 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:40.909507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 660/1323] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:40.915237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f1f4-7cd7-4a38-b775-b856a57f122b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:40.915690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f1f4-7cd7-4a38-b775-b856a57f122b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:40.915781 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f1f4-7cd7-4a38-b775-b856a57f122b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:41.002136 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5fd5f1f4-7cd7-4a38-b775-b856a57f122b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:41.004039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5fd5f1f4-7cd7-4a38-b775-b856a57f122b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:41.080991 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:41.120954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 664/1324] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:41.171167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c67e9-a9e7-490b-b1c6-f4662daad851 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:29:41.172804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cf4c67e9-a9e7-490b-b1c6-f4662daad851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:41.296463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c67e9-a9e7-490b-b1c6-f4662daad851 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:29:41.313565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:41.401731 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661741f-a749-4cdc-87ed-e156cc519d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:29:41.402370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661741f-a749-4cdc-87ed-e156cc519d94 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:41.402645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661741f-a749-4cdc-87ed-e156cc519d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:41.402969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661741f-a749-4cdc-87ed-e156cc519d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:41.405121 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6661741f-a749-4cdc-87ed-e156cc519d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:41.524124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661741f-a749-4cdc-87ed-e156cc519d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:29:41.525602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661741f-a749-4cdc-87ed-e156cc519d94 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:29:41.544483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 661/1325] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:29:43.615807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d913f-214e-40d9-9b69-b26552c03958 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:43.616396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d913f-214e-40d9-9b69-b26552c03958 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:43.616396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d913f-214e-40d9-9b69-b26552c03958 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:43.617734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 665/1326] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:43.622972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e0b74-33cd-47d2-9263-503036bd7a9d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:43.624271 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e0b74-33cd-47d2-9263-503036bd7a9d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:43.624271 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e0b74-33cd-47d2-9263-503036bd7a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:43.673715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:43.692412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ff7e0b74-33cd-47d2-9263-503036bd7a9d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:43.693705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ff7e0b74-33cd-47d2-9263-503036bd7a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:43.790454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 662/1327] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:43.827164 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d913f-214e-40d9-9b69-b26552c03958 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:29:43.828777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bc0d913f-214e-40d9-9b69-b26552c03958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:43.932865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d913f-214e-40d9-9b69-b26552c03958 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:29:44.008320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e507d05-5b86-4d45-9acb-bca7691ed0ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:29:44.008942 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507d05-5b86-4d45-9acb-bca7691ed0ec service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:44.009224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507d05-5b86-4d45-9acb-bca7691ed0ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:44.009498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507d05-5b86-4d45-9acb-bca7691ed0ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:44.011128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1e507d05-5b86-4d45-9acb-bca7691ed0ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:44.029234 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:44.127179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e507d05-5b86-4d45-9acb-bca7691ed0ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:29:44.128672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e507d05-5b86-4d45-9acb-bca7691ed0ec service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:29:44.149725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 666/1328] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:29:46.214006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d52ec7-0af6-42ee-859d-61861f252306 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:46.214527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d52ec7-0af6-42ee-859d-61861f252306 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:46.214527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d52ec7-0af6-42ee-859d-61861f252306 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:46.215797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 663/1329] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:46.221865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc53d3f-03a3-4fec-a4ef-a740288a5943 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:46.222331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc53d3f-03a3-4fec-a4ef-a740288a5943 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:46.222433 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc53d3f-03a3-4fec-a4ef-a740288a5943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:46.288579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-dcc53d3f-03a3-4fec-a4ef-a740288a5943 None None] MFA Rules not processed for user `3d8ca4abbdac4f939233a06b4533d187`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:46.289818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dcc53d3f-03a3-4fec-a4ef-a740288a5943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:46.353632 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:46.387281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 667/1330] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:29:46 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:46.423385 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d52ec7-0af6-42ee-859d-61861f252306 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:29:46.424952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-99d52ec7-0af6-42ee-859d-61861f252306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:46.535288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d52ec7-0af6-42ee-859d-61861f252306 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:29:46.555806 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:46.611283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c37c07-3c32-4aee-a31d-89e16a7c1462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:29:46.611888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37c07-3c32-4aee-a31d-89e16a7c1462 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:46.612196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37c07-3c32-4aee-a31d-89e16a7c1462 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:46.612469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37c07-3c32-4aee-a31d-89e16a7c1462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:46.614053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f1c37c07-3c32-4aee-a31d-89e16a7c1462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:46.722653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c37c07-3c32-4aee-a31d-89e16a7c1462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:29:46.724148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c37c07-3c32-4aee-a31d-89e16a7c1462 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:29:46.743418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 664/1331] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:29:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:29:49.210182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67255b87-2724-4a28-8007-ccf233703815 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:49.210706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67255b87-2724-4a28-8007-ccf233703815 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:49.210706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67255b87-2724-4a28-8007-ccf233703815 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:49.212031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 668/1332] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:49.217298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89401c7-1b32-42ac-a914-68be14f8e86c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:49.217804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89401c7-1b32-42ac-a914-68be14f8e86c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:49.217991 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89401c7-1b32-42ac-a914-68be14f8e86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:49.283999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:49.286711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d89401c7-1b32-42ac-a914-68be14f8e86c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:49.287963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d89401c7-1b32-42ac-a914-68be14f8e86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:49.388465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 665/1333] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:49.424859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67255b87-2724-4a28-8007-ccf233703815 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:29:49.426422 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-67255b87-2724-4a28-8007-ccf233703815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:49.528399 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67255b87-2724-4a28-8007-ccf233703815 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:29:49.600558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4f57f-1795-4791-9251-6548698f7a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:29:49.601147 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4f57f-1795-4791-9251-6548698f7a60 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:49.601420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4f57f-1795-4791-9251-6548698f7a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:49.601687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4f57f-1795-4791-9251-6548698f7a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:49.603271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5aa4f57f-1795-4791-9251-6548698f7a60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:49.629992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:49.711189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4f57f-1795-4791-9251-6548698f7a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:29:49.712659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4f57f-1795-4791-9251-6548698f7a60 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:29:49.731931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 669/1334] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:29:51.777932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce822d-834c-4668-bde4-28e64b914f43 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:51.778521 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce822d-834c-4668-bde4-28e64b914f43 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:51.778521 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce822d-834c-4668-bde4-28e64b914f43 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:51.780407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 666/1335] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:51.785430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13999783-c8ec-43c4-80e7-602029820b7a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:51.785893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13999783-c8ec-43c4-80e7-602029820b7a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:51.786000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13999783-c8ec-43c4-80e7-602029820b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:51.851104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-13999783-c8ec-43c4-80e7-602029820b7a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:51.852345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-13999783-c8ec-43c4-80e7-602029820b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:51.926213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:51.948475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 670/1336] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:51.990169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce822d-834c-4668-bde4-28e64b914f43 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:29:51.991711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d4ce822d-834c-4668-bde4-28e64b914f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:52.095108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce822d-834c-4668-bde4-28e64b914f43 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:29:52.123277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:52.168594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0481f-a701-4866-83fb-26c4b74a5925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:29:52.169198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0481f-a701-4866-83fb-26c4b74a5925 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:52.169503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0481f-a701-4866-83fb-26c4b74a5925 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:52.169806 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0481f-a701-4866-83fb-26c4b74a5925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:52.171416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ccf0481f-a701-4866-83fb-26c4b74a5925 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:52.277150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0481f-a701-4866-83fb-26c4b74a5925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:29:52.278598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0481f-a701-4866-83fb-26c4b74a5925 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:29:52.297983 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 667/1337] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:29:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:29:55.006355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adbc55-5235-4329-800f-3aa0044d7b43 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:55.006872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adbc55-5235-4329-800f-3aa0044d7b43 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:55.006872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adbc55-5235-4329-800f-3aa0044d7b43 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:55.008293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 671/1338] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:55.013389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87977b41-11e8-43cd-bbec-cfd684cc3e7b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:55.013816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87977b41-11e8-43cd-bbec-cfd684cc3e7b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:55.014016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87977b41-11e8-43cd-bbec-cfd684cc3e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:55.079613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-87977b41-11e8-43cd-bbec-cfd684cc3e7b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:55.080852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-87977b41-11e8-43cd-bbec-cfd684cc3e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:55.088868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:55.174945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 668/1339] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:29:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:55.211598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adbc55-5235-4329-800f-3aa0044d7b43 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:29:55.213184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e6adbc55-5235-4329-800f-3aa0044d7b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:55.316609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adbc55-5235-4329-800f-3aa0044d7b43 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:29:55.390696 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d076ea-e0f3-436a-90d9-885437e24462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:29:55.391290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d076ea-e0f3-436a-90d9-885437e24462 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:55.391564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d076ea-e0f3-436a-90d9-885437e24462 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:55.391830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d076ea-e0f3-436a-90d9-885437e24462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:55.393439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-67d076ea-e0f3-436a-90d9-885437e24462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:55.428224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:55.500766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d076ea-e0f3-436a-90d9-885437e24462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:29:55.502382 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d076ea-e0f3-436a-90d9-885437e24462 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:29:55.520511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 672/1340] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:29:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:29:59.722024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499f6c-6e7b-4f23-81bb-41babededc66 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:59.722435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499f6c-6e7b-4f23-81bb-41babededc66 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:59.722605 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499f6c-6e7b-4f23-81bb-41babededc66 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:59.723961 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 669/1341] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:29:59.729511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec1a3e-da54-4c43-84e7-0492d4e80a99 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:29:59.729935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec1a3e-da54-4c43-84e7-0492d4e80a99 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:29:59.730067 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec1a3e-da54-4c43-84e7-0492d4e80a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:29:59.797217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c9ec1a3e-da54-4c43-84e7-0492d4e80a99 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:29:59.798438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c9ec1a3e-da54-4c43-84e7-0492d4e80a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:29:59.884196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:29:59.895939 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 673/1342] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:29:59.935567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad499f6c-6e7b-4f23-81bb-41babededc66 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:29:59.937227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ad499f6c-6e7b-4f23-81bb-41babededc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:00.045753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad499f6c-6e7b-4f23-81bb-41babededc66 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:00.088050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:00.124048 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002862f9-63c9-473f-8777-22ea5bba9c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:00.124653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002862f9-63c9-473f-8777-22ea5bba9c76 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:00.124939 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002862f9-63c9-473f-8777-22ea5bba9c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:00.125214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002862f9-63c9-473f-8777-22ea5bba9c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:00.127088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-002862f9-63c9-473f-8777-22ea5bba9c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:00.238563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002862f9-63c9-473f-8777-22ea5bba9c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:00.239985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002862f9-63c9-473f-8777-22ea5bba9c76 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:00.258988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 670/1343] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:29:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:03.186069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3aba34-6358-46d8-9b51-39704e2f90bc None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:03.186604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3aba34-6358-46d8-9b51-39704e2f90bc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:03.186604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3aba34-6358-46d8-9b51-39704e2f90bc None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:03.188048 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 674/1344] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:03.193515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2ad0-ad6f-49bf-8c50-ef71be2a1fe8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:03.193946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2ad0-ad6f-49bf-8c50-ef71be2a1fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:03.194050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2ad0-ad6f-49bf-8c50-ef71be2a1fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:03.259677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0f7a2ad0-ad6f-49bf-8c50-ef71be2a1fe8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:03.260903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0f7a2ad0-ad6f-49bf-8c50-ef71be2a1fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:03.279065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:03.358091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 671/1345] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:03.396218 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3aba34-6358-46d8-9b51-39704e2f90bc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:03.397849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bc3aba34-6358-46d8-9b51-39704e2f90bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:03.504377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3aba34-6358-46d8-9b51-39704e2f90bc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:03.578975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25180a-b831-4836-b9a9-5e7523fd49aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:03.579591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25180a-b831-4836-b9a9-5e7523fd49aa service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:03.579869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25180a-b831-4836-b9a9-5e7523fd49aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:03.580174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25180a-b831-4836-b9a9-5e7523fd49aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:03.581784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7b25180a-b831-4836-b9a9-5e7523fd49aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:03.626822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:03.829331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25180a-b831-4836-b9a9-5e7523fd49aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:03.830785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25180a-b831-4836-b9a9-5e7523fd49aa service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:03.850325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 675/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:08.118585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200212ed-7c1d-4e5d-ab4d-d87b427a9d44 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:08.119150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200212ed-7c1d-4e5d-ab4d-d87b427a9d44 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:08.119252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200212ed-7c1d-4e5d-ab4d-d87b427a9d44 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:08.120733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 672/1347] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:08.126864 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472d8d1-d0be-46b1-a2a6-361f5e207577 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:08.127315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472d8d1-d0be-46b1-a2a6-361f5e207577 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:08.127407 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472d8d1-d0be-46b1-a2a6-361f5e207577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:08.198359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3472d8d1-d0be-46b1-a2a6-361f5e207577 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:08.199697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3472d8d1-d0be-46b1-a2a6-361f5e207577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:08.304956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:08.307705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 676/1348] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:08.347283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200212ed-7c1d-4e5d-ab4d-d87b427a9d44 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:08.348920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-200212ed-7c1d-4e5d-ab4d-d87b427a9d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:08.460661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200212ed-7c1d-4e5d-ab4d-d87b427a9d44 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:08.510498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:08.538117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0aae4-fda8-4501-bb78-9731a52c1301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:08.538763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0aae4-fda8-4501-bb78-9731a52c1301 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:08.539044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0aae4-fda8-4501-bb78-9731a52c1301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:08.539513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0aae4-fda8-4501-bb78-9731a52c1301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:08.541592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-81c0aae4-fda8-4501-bb78-9731a52c1301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:08.654824 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0aae4-fda8-4501-bb78-9731a52c1301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:08.656298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0aae4-fda8-4501-bb78-9731a52c1301 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:08.675465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 673/1349] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:14.007470 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e5ea4-d949-4d21-a04c-10965192408f None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:14.009769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e5ea4-d949-4d21-a04c-10965192408f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:14.010718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e5ea4-d949-4d21-a04c-10965192408f None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:14.015604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 677/1350] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:30:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:14.022295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1be5f-bbc1-42ec-bbe2-6eb276f9793a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:14.023266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1be5f-bbc1-42ec-bbe2-6eb276f9793a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:14.023645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1be5f-bbc1-42ec-bbe2-6eb276f9793a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:14.199040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e6a1be5f-bbc1-42ec-bbe2-6eb276f9793a None None] MFA Rules not processed for user `db1d7e39e8c14f5db40efa743fc3b652`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:14.200824 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e6a1be5f-bbc1-42ec-bbe2-6eb276f9793a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:14.231253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:14.407776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 674/1351] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:30:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:18.668436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a6737-9b58-4f62-9043-43258da28c76 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:18.669458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a6737-9b58-4f62-9043-43258da28c76 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:18.669682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a6737-9b58-4f62-9043-43258da28c76 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:18.671281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 678/1352] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:30:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:18.676343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89cf02-e8bd-4a65-a6be-40eff4922e99 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:18.676759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89cf02-e8bd-4a65-a6be-40eff4922e99 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:18.676903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89cf02-e8bd-4a65-a6be-40eff4922e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:18.759678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ee89cf02-e8bd-4a65-a6be-40eff4922e99 None None] MFA Rules not processed for user `db1d7e39e8c14f5db40efa743fc3b652`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:18.761895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ee89cf02-e8bd-4a65-a6be-40eff4922e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:18.845937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:18.888188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 675/1353] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:30:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:20.291638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32bf19-1654-472a-af30-6cbb2a047fe0 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:20.292334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32bf19-1654-472a-af30-6cbb2a047fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:20.292334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32bf19-1654-472a-af30-6cbb2a047fe0 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:20.294491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 679/1354] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:30:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:20.299570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee871b-f857-4fd3-98a9-e7ab87e873f9 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:20.300068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee871b-f857-4fd3-98a9-e7ab87e873f9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:20.300177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee871b-f857-4fd3-98a9-e7ab87e873f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:20.377495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c8ee871b-f857-4fd3-98a9-e7ab87e873f9 None None] MFA Rules not processed for user `db1d7e39e8c14f5db40efa743fc3b652`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:20.378730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c8ee871b-f857-4fd3-98a9-e7ab87e873f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:20.494734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 676/1355] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:30:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:30:23.769168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4e602-6a83-4884-9100-9abefc15bf65 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:23.770061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4e602-6a83-4884-9100-9abefc15bf65 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:23.780371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4e602-6a83-4884-9100-9abefc15bf65 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:23.780371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 680/1356] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:23.780567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c053d8a-08bc-431e-9267-fa54e87f80b9 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:23.780567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c053d8a-08bc-431e-9267-fa54e87f80b9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:23.780567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c053d8a-08bc-431e-9267-fa54e87f80b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:23.855053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:23.893712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9c053d8a-08bc-431e-9267-fa54e87f80b9 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:23.895280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9c053d8a-08bc-431e-9267-fa54e87f80b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:24.019057 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 677/1357] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:24.072419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae863272-a790-4abf-b8ef-6adc70872f2b None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:24.073466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae863272-a790-4abf-b8ef-6adc70872f2b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:24.074053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae863272-a790-4abf-b8ef-6adc70872f2b None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:24.076234 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 681/1358] 10.222.0.30 () {54 vars in 962 bytes} [Sun Aug 7 22:30:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:24.086188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519d89b-83f8-48ab-adbf-0c6e6226ec42 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:24.087026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519d89b-83f8-48ab-adbf-0c6e6226ec42 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:24.087579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519d89b-83f8-48ab-adbf-0c6e6226ec42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:24.176032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4519d89b-83f8-48ab-adbf-0c6e6226ec42 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:24.177374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4519d89b-83f8-48ab-adbf-0c6e6226ec42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:24.231063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:24.307711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 678/1359] 10.222.0.30 () {58 vars in 1059 bytes} [Sun Aug 7 22:30:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:24.316182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae863272-a790-4abf-b8ef-6adc70872f2b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:24.317680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ae863272-a790-4abf-b8ef-6adc70872f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:24.475717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae863272-a790-4abf-b8ef-6adc70872f2b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:24.560237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:24.603021 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d690ff4-0b53-449a-84c8-f04a26501c6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:24.603619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d690ff4-0b53-449a-84c8-f04a26501c6c service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:24.604058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d690ff4-0b53-449a-84c8-f04a26501c6c service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:24.604305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d690ff4-0b53-449a-84c8-f04a26501c6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:24.606139 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8d690ff4-0b53-449a-84c8-f04a26501c6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:24.703264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d690ff4-0b53-449a-84c8-f04a26501c6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:24.704913 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d690ff4-0b53-449a-84c8-f04a26501c6c service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:24.723172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 682/1360] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:27.058208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6260a-62a4-46aa-8ea0-a7636ee28219 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:27.058728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6260a-62a4-46aa-8ea0-a7636ee28219 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:27.058728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6260a-62a4-46aa-8ea0-a7636ee28219 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:27.060134 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 679/1361] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:27.065722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65a93c-3dd5-401f-815a-c6e0f60b5b5a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:27.066180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65a93c-3dd5-401f-815a-c6e0f60b5b5a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:27.066275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65a93c-3dd5-401f-815a-c6e0f60b5b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:27.141331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1c65a93c-3dd5-401f-815a-c6e0f60b5b5a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:27.142550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1c65a93c-3dd5-401f-815a-c6e0f60b5b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:27.183100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:27.252158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 683/1362] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:27.296214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6260a-62a4-46aa-8ea0-a7636ee28219 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:27.298425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3ef6260a-62a4-46aa-8ea0-a7636ee28219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:27.467080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6260a-62a4-46aa-8ea0-a7636ee28219 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:27.477189 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:27.605153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b40ef-3835-4976-ba38-6327612cd6e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:27.605786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b40ef-3835-4976-ba38-6327612cd6e4 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:27.606061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b40ef-3835-4976-ba38-6327612cd6e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:27.606334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b40ef-3835-4976-ba38-6327612cd6e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:27.607948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-dc2b40ef-3835-4976-ba38-6327612cd6e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:27.708828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b40ef-3835-4976-ba38-6327612cd6e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:27.710526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b40ef-3835-4976-ba38-6327612cd6e4 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:27.732205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 680/1363] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:30:30.303246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303ca32-a889-4637-9ffe-61495f972638 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:30.304198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303ca32-a889-4637-9ffe-61495f972638 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:30.304633 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303ca32-a889-4637-9ffe-61495f972638 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:30.306143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 684/1364] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:30.312100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de06587-def1-4ccb-9b92-20b8483a1714 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:30.312867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de06587-def1-4ccb-9b92-20b8483a1714 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:30.312867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de06587-def1-4ccb-9b92-20b8483a1714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:30.340603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:30.437973 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4de06587-def1-4ccb-9b92-20b8483a1714 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:30.438912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4de06587-def1-4ccb-9b92-20b8483a1714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:30.558645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 681/1365] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:30:30.608409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d7f45-0a3f-4f59-b6d4-96736295a9e6 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:30.608918 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d7f45-0a3f-4f59-b6d4-96736295a9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:30.609366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d7f45-0a3f-4f59-b6d4-96736295a9e6 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:30.611090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 685/1366] 10.222.0.30 () {54 vars in 962 bytes} [Sun Aug 7 22:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:30.620733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011a66f-4045-4bb2-a6fd-3b1df19c4370 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:30.621169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011a66f-4045-4bb2-a6fd-3b1df19c4370 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:30.621696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011a66f-4045-4bb2-a6fd-3b1df19c4370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:30.729604 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a011a66f-4045-4bb2-a6fd-3b1df19c4370 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:30.733582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a011a66f-4045-4bb2-a6fd-3b1df19c4370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:30.772964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:30.885856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 682/1367] 10.222.0.30 () {58 vars in 1059 bytes} [Sun Aug 7 22:30:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:30.897285 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d7f45-0a3f-4f59-b6d4-96736295a9e6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:30.898823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c06d7f45-0a3f-4f59-b6d4-96736295a9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:31.091271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d7f45-0a3f-4f59-b6d4-96736295a9e6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:31.114518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:31.242590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd8b73-8ba2-45d9-aced-dec0b2e6f734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:31.243163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd8b73-8ba2-45d9-aced-dec0b2e6f734 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:31.243448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd8b73-8ba2-45d9-aced-dec0b2e6f734 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:31.243760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd8b73-8ba2-45d9-aced-dec0b2e6f734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:31.245794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d4bd8b73-8ba2-45d9-aced-dec0b2e6f734 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:31.366527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd8b73-8ba2-45d9-aced-dec0b2e6f734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:31.369395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd8b73-8ba2-45d9-aced-dec0b2e6f734 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:31.397226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 686/1368] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:32.292350 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a310c84-7d5e-4b45-a29b-9b6dbafa7b21 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:32.299578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a310c84-7d5e-4b45-a29b-9b6dbafa7b21 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:32.299578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a310c84-7d5e-4b45-a29b-9b6dbafa7b21 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:32.299578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 683/1369] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:32.300234 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15219bb-2ae7-43fd-bb0a-1aa299a482c7 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:32.300529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15219bb-2ae7-43fd-bb0a-1aa299a482c7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:32.300808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15219bb-2ae7-43fd-bb0a-1aa299a482c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:32.353138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:32.405069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c15219bb-2ae7-43fd-bb0a-1aa299a482c7 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:32.407092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c15219bb-2ae7-43fd-bb0a-1aa299a482c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:32.531996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 687/1370] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:32.890720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9b666-5487-44e7-abd4-8bb523a3c618 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:32.890720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9b666-5487-44e7-abd4-8bb523a3c618 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:32.890720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9b666-5487-44e7-abd4-8bb523a3c618 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:32.894126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 684/1371] 10.222.0.30 () {54 vars in 964 bytes} [Sun Aug 7 22:30:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:32.900452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d1d3-5214-445d-b052-bdbda6c39b77 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:32.900817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d1d3-5214-445d-b052-bdbda6c39b77 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:32.901169 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d1d3-5214-445d-b052-bdbda6c39b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:33.022394 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e2c1d1d3-5214-445d-b052-bdbda6c39b77 None None] MFA Rules not processed for user `407b8483d4f34be19d2c7c7b6013b313`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:33.023947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e2c1d1d3-5214-445d-b052-bdbda6c39b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:33.080241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:33.125388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 688/1372] 10.222.0.30 () {58 vars in 1061 bytes} [Sun Aug 7 22:30:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:33.126211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9b666-5487-44e7-abd4-8bb523a3c618 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:33.128044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4cd9b666-5487-44e7-abd4-8bb523a3c618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:33.258317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9b666-5487-44e7-abd4-8bb523a3c618 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:33.314506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:33.357191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864af54-a014-4612-9d23-df78a08852f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:33.358211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864af54-a014-4612-9d23-df78a08852f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:33.358705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864af54-a014-4612-9d23-df78a08852f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:33.359266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864af54-a014-4612-9d23-df78a08852f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:33.361360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1864af54-a014-4612-9d23-df78a08852f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:33.493773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864af54-a014-4612-9d23-df78a08852f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:33.495844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864af54-a014-4612-9d23-df78a08852f2 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:33.519266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 685/1373] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:33.739388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff017-e671-4cbf-9179-1bda503e0e56 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:33.739823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff017-e671-4cbf-9179-1bda503e0e56 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:33.740061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff017-e671-4cbf-9179-1bda503e0e56 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:33.746685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7152c-7598-46ed-a414-ffeee517691f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:33.746919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7152c-7598-46ed-a414-ffeee517691f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:33.747165 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7152c-7598-46ed-a414-ffeee517691f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:33.747691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 689/1374] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:33.877957 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a3d7152c-7598-46ed-a414-ffeee517691f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:33.879327 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a3d7152c-7598-46ed-a414-ffeee517691f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:33.916615 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:34.055748 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 686/1375] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:30:34.102520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ff017-e671-4cbf-9179-1bda503e0e56 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:34.104426 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3a4ff017-e671-4cbf-9179-1bda503e0e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:34.276204 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ff017-e671-4cbf-9179-1bda503e0e56 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:34.330580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:34.398185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dce8b-59c9-4205-bcc9-54be4f12f012 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:34.398895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dce8b-59c9-4205-bcc9-54be4f12f012 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:34.399241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dce8b-59c9-4205-bcc9-54be4f12f012 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:34.399515 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dce8b-59c9-4205-bcc9-54be4f12f012 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:34.401150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f69dce8b-59c9-4205-bcc9-54be4f12f012 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:34.501691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69dce8b-59c9-4205-bcc9-54be4f12f012 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:34.503394 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69dce8b-59c9-4205-bcc9-54be4f12f012 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:34.523096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 690/1376] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:36.754826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b05443-be2b-456e-a9c1-b3371738baf0 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:36.755330 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b05443-be2b-456e-a9c1-b3371738baf0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:36.755428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b05443-be2b-456e-a9c1-b3371738baf0 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:36.756773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 687/1377] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:36.762090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8e324-c9ec-461a-9242-4cd8cac718db None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:36.762488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8e324-c9ec-461a-9242-4cd8cac718db None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:36.762665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8e324-c9ec-461a-9242-4cd8cac718db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:36.853709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-60e8e324-c9ec-461a-9242-4cd8cac718db None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:36.854983 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-60e8e324-c9ec-461a-9242-4cd8cac718db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:36.863577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:37.000525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 691/1378] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:37.045692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b05443-be2b-456e-a9c1-b3371738baf0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:37.047270 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-01b05443-be2b-456e-a9c1-b3371738baf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:37.259043 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b05443-be2b-456e-a9c1-b3371738baf0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:37.261727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:37.466413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab45f8-287a-4194-aca7-871bf57f727d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:37.467564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab45f8-287a-4194-aca7-871bf57f727d service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:37.468080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab45f8-287a-4194-aca7-871bf57f727d service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:37.468560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab45f8-287a-4194-aca7-871bf57f727d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:37.470386 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-17ab45f8-287a-4194-aca7-871bf57f727d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:37.575420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab45f8-287a-4194-aca7-871bf57f727d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:37.577450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab45f8-287a-4194-aca7-871bf57f727d service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:37.599714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 688/1379] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:39.827274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d8161-0ebf-4654-805b-821b0739ac55 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:39.827812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d8161-0ebf-4654-805b-821b0739ac55 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:39.827812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d8161-0ebf-4654-805b-821b0739ac55 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:39.829383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 692/1380] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:39.834408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dfedb-8893-4475-8a32-ff98a60352c0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:39.834895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dfedb-8893-4475-8a32-ff98a60352c0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:39.834978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dfedb-8893-4475-8a32-ff98a60352c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:39.846627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:39.914864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ff4dfedb-8893-4475-8a32-ff98a60352c0 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:39.916230 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ff4dfedb-8893-4475-8a32-ff98a60352c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:40.027224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 689/1381] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:40.071607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d8161-0ebf-4654-805b-821b0739ac55 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:40.073236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ea1d8161-0ebf-4654-805b-821b0739ac55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:40.217622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:40.227735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d8161-0ebf-4654-805b-821b0739ac55 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:40.352686 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18d795-d27c-4b12-9060-22200f68699c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:40.353215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18d795-d27c-4b12-9060-22200f68699c service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:40.353489 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18d795-d27c-4b12-9060-22200f68699c service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:40.353759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18d795-d27c-4b12-9060-22200f68699c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:40.355565 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4d18d795-d27c-4b12-9060-22200f68699c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:40.468772 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18d795-d27c-4b12-9060-22200f68699c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:40.470394 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18d795-d27c-4b12-9060-22200f68699c service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:40.481540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:40.490063 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 693/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:43.222041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02854c7f-9fbb-4e60-be97-12f31f870468 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:43.222567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02854c7f-9fbb-4e60-be97-12f31f870468 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:43.222660 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02854c7f-9fbb-4e60-be97-12f31f870468 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:43.224421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 690/1383] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:43.231037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6088020-6c59-44d2-8e06-ef733905c65e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:43.231448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6088020-6c59-44d2-8e06-ef733905c65e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:43.231586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6088020-6c59-44d2-8e06-ef733905c65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:43.299047 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b6088020-6c59-44d2-8e06-ef733905c65e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:43.300298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b6088020-6c59-44d2-8e06-ef733905c65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:43.407810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 694/1384] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:43.452714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02854c7f-9fbb-4e60-be97-12f31f870468 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:43.454316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-02854c7f-9fbb-4e60-be97-12f31f870468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:43.546499 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:43.611606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02854c7f-9fbb-4e60-be97-12f31f870468 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:43.725825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9ab4c-64be-4d9a-bb1d-bd3aad1ee335 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:43.726690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9ab4c-64be-4d9a-bb1d-bd3aad1ee335 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:43.727120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9ab4c-64be-4d9a-bb1d-bd3aad1ee335 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:43.727478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9ab4c-64be-4d9a-bb1d-bd3aad1ee335 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:43.729587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-31e9ab4c-64be-4d9a-bb1d-bd3aad1ee335 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:43.790379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:43.841281 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9ab4c-64be-4d9a-bb1d-bd3aad1ee335 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:43.843293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9ab4c-64be-4d9a-bb1d-bd3aad1ee335 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:43.869379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 691/1385] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:45.937987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb10e2-9fd0-4ac3-b55b-9685f2978aae None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:45.938661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb10e2-9fd0-4ac3-b55b-9685f2978aae None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:45.938661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb10e2-9fd0-4ac3-b55b-9685f2978aae None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:45.940100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 695/1386] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:45.946503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fdf2a-6937-4e17-a13b-c43b9bb05907 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:45.947452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fdf2a-6937-4e17-a13b-c43b9bb05907 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:45.948038 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fdf2a-6937-4e17-a13b-c43b9bb05907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:46.030562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d27fdf2a-6937-4e17-a13b-c43b9bb05907 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:46.031821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d27fdf2a-6937-4e17-a13b-c43b9bb05907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:46.135149 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 692/1387] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:46.193211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb10e2-9fd0-4ac3-b55b-9685f2978aae None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:46.194755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-85bb10e2-9fd0-4ac3-b55b-9685f2978aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:46.279171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:46.351026 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb10e2-9fd0-4ac3-b55b-9685f2978aae None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:46.474066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbedcfc-d441-4a93-aa76-4c3124daed3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:46.474590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbedcfc-d441-4a93-aa76-4c3124daed3d service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:46.474865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbedcfc-d441-4a93-aa76-4c3124daed3d service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:46.475130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbedcfc-d441-4a93-aa76-4c3124daed3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:46.476807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1cbedcfc-d441-4a93-aa76-4c3124daed3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:46.517835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:46.571999 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbedcfc-d441-4a93-aa76-4c3124daed3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:46.573683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbedcfc-d441-4a93-aa76-4c3124daed3d service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:46.593526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 696/1388] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:48.812597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6fe24-db58-4f9b-83f5-88d475ca5621 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:48.813173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6fe24-db58-4f9b-83f5-88d475ca5621 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:48.813173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6fe24-db58-4f9b-83f5-88d475ca5621 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:48.814499 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 693/1389] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:48.819810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f18d7-1259-46a1-adfd-c81472bf6878 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:48.820297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f18d7-1259-46a1-adfd-c81472bf6878 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:48.820396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f18d7-1259-46a1-adfd-c81472bf6878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:48.887233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5d8f18d7-1259-46a1-adfd-c81472bf6878 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:48.888484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5d8f18d7-1259-46a1-adfd-c81472bf6878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:48.988447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 697/1390] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:49.033931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6fe24-db58-4f9b-83f5-88d475ca5621 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:49.035450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-acf6fe24-db58-4f9b-83f5-88d475ca5621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:49.066967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:49.182324 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6fe24-db58-4f9b-83f5-88d475ca5621 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:49.279423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f50e6-ebe8-4bd8-91e2-15eeb68c4045 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:49.279710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f50e6-ebe8-4bd8-91e2-15eeb68c4045 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:49.279982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f50e6-ebe8-4bd8-91e2-15eeb68c4045 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:49.281364 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 698/1391] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:49.286194 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68b7e4-a39d-4592-93db-9bceaa4a99a8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:49.286472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68b7e4-a39d-4592-93db-9bceaa4a99a8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:49.286749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68b7e4-a39d-4592-93db-9bceaa4a99a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:49.287118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:49.299651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c654f7-94c6-4d1e-ade6-c73bd1386120 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:49.300402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c654f7-94c6-4d1e-ade6-c73bd1386120 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:49.300685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c654f7-94c6-4d1e-ade6-c73bd1386120 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:49.300958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c654f7-94c6-4d1e-ade6-c73bd1386120 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:49.302245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:49.302585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a2c654f7-94c6-4d1e-ade6-c73bd1386120 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:49.354413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-aa68b7e4-a39d-4592-93db-9bceaa4a99a8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:49.356267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-aa68b7e4-a39d-4592-93db-9bceaa4a99a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:49.396394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c654f7-94c6-4d1e-ade6-c73bd1386120 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:49.397985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c654f7-94c6-4d1e-ade6-c73bd1386120 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:49.416093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 694/1392] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:49.456279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 699/1393] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:30:49.514118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2d932-7b27-4d98-b36c-fb8e89d5de2c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:49.514412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2d932-7b27-4d98-b36c-fb8e89d5de2c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:49.514692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2d932-7b27-4d98-b36c-fb8e89d5de2c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:49.516198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 695/1394] 10.222.0.30 () {54 vars in 966 bytes} [Sun Aug 7 22:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:49.521232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d1cd2-d7c4-4a08-a0e2-07a8635b23c1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:49.521520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d1cd2-d7c4-4a08-a0e2-07a8635b23c1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:49.521796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d1cd2-d7c4-4a08-a0e2-07a8635b23c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:49.596361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-707d1cd2-d7c4-4a08-a0e2-07a8635b23c1 None None] MFA Rules not processed for user `cfb2129c006f4ddeba7d128a8d43d5de`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:49.597589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-707d1cd2-d7c4-4a08-a0e2-07a8635b23c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:49.637209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:49.684066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 700/1395] 10.222.0.30 () {58 vars in 1063 bytes} [Sun Aug 7 22:30:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:49.691645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2d932-7b27-4d98-b36c-fb8e89d5de2c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:49.693506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-47f2d932-7b27-4d98-b36c-fb8e89d5de2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:49.804917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2d932-7b27-4d98-b36c-fb8e89d5de2c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:49.819389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:49.880483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ef5c0-a1b5-4434-afa5-e446f6f7a07c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb2129c006f4ddeba7d128a8d43d5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:49.882643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ef5c0-a1b5-4434-afa5-e446f6f7a07c service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:49.882643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ef5c0-a1b5-4434-afa5-e446f6f7a07c service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:49.882643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ef5c0-a1b5-4434-afa5-e446f6f7a07c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:49.883244 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d20ef5c0-a1b5-4434-afa5-e446f6f7a07c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:49.996776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ef5c0-a1b5-4434-afa5-e446f6f7a07c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:49.998246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ef5c0-a1b5-4434-afa5-e446f6f7a07c service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:50.017203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 696/1396] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:50.956262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b10d-7143-4106-9d00-f95e116246bc None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:50.956833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b10d-7143-4106-9d00-f95e116246bc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:50.956959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b10d-7143-4106-9d00-f95e116246bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:51.057441 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-82d0b10d-7143-4106-9d00-f95e116246bc None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:51.059655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-82d0b10d-7143-4106-9d00-f95e116246bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:51.149480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 701/1397] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:30:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:51.193108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a895a4-936f-4013-8574-280838689b16 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:51.193589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a895a4-936f-4013-8574-280838689b16 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:51.193589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a895a4-936f-4013-8574-280838689b16 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:51.194951 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 697/1398] 10.222.0.30 () {54 vars in 943 bytes} [Sun Aug 7 22:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:51.202310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91dc2-c4e1-4cb2-bad9-5f1438728d44 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:51.202795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91dc2-c4e1-4cb2-bad9-5f1438728d44 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:51.203074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91dc2-c4e1-4cb2-bad9-5f1438728d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:51.262677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:51.280760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b9d91dc2-c4e1-4cb2-bad9-5f1438728d44 None None] MFA Rules not processed for user `e86991778c2b47c58f10d9d9e3ca5e82`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:51.281964 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b9d91dc2-c4e1-4cb2-bad9-5f1438728d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:51.392209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 702/1399] 10.222.0.30 () {58 vars in 1040 bytes} [Sun Aug 7 22:30:51 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:51.401794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a895a4-936f-4013-8574-280838689b16 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:51.403372 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-94a895a4-936f-4013-8574-280838689b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:51.449654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:51.541576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a895a4-936f-4013-8574-280838689b16 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:51.624779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34709cd7-3fee-40ba-ab02-3f63821552e2 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:51.625058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34709cd7-3fee-40ba-ab02-3f63821552e2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:51.625423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34709cd7-3fee-40ba-ab02-3f63821552e2 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:51.626919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 703/1400] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:51.631776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa82ba-4c14-4193-8ed8-3fc61d3f7eea None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:51.632079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa82ba-4c14-4193-8ed8-3fc61d3f7eea None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:51.632358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa82ba-4c14-4193-8ed8-3fc61d3f7eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:51.646775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56529531-3cff-49bd-8fb2-05ceb002bdfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:51.647390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56529531-3cff-49bd-8fb2-05ceb002bdfc service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:51.647673 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56529531-3cff-49bd-8fb2-05ceb002bdfc service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:51.647943 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56529531-3cff-49bd-8fb2-05ceb002bdfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:51.649616 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-56529531-3cff-49bd-8fb2-05ceb002bdfc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:51.679036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:51.700146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-59aa82ba-4c14-4193-8ed8-3fc61d3f7eea None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:51.701278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-59aa82ba-4c14-4193-8ed8-3fc61d3f7eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:51.734549 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56529531-3cff-49bd-8fb2-05ceb002bdfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:51.736143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56529531-3cff-49bd-8fb2-05ceb002bdfc service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:51.744172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 698/1401] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:30:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:30:51.757571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:51.799616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 704/1402] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:30:51.845490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56529531-3cff-49bd-8fb2-05ceb002bdfc service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:51.846700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-56529531-3cff-49bd-8fb2-05ceb002bdfc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:51.938978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56529531-3cff-49bd-8fb2-05ceb002bdfc service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:52.008888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:52.056512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809487d-36f5-4925-947f-2abbb3250115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:52.057117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809487d-36f5-4925-947f-2abbb3250115 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:52.057394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809487d-36f5-4925-947f-2abbb3250115 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:52.057662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809487d-36f5-4925-947f-2abbb3250115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:52.060043 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1809487d-36f5-4925-947f-2abbb3250115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:52.151259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809487d-36f5-4925-947f-2abbb3250115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:52.153098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809487d-36f5-4925-947f-2abbb3250115 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:52.176253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 699/1403] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:30:54.398228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7172a13-eea0-4b83-b49d-5315f045039f None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:54.398758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7172a13-eea0-4b83-b49d-5315f045039f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:54.398758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7172a13-eea0-4b83-b49d-5315f045039f None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:54.400328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 705/1404] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:54.409068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab5ad3-1a5b-4757-ac1e-ed9e73565904 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:54.410009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab5ad3-1a5b-4757-ac1e-ed9e73565904 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:54.410451 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab5ad3-1a5b-4757-ac1e-ed9e73565904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:54.510674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-dfab5ad3-1a5b-4757-ac1e-ed9e73565904 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:54.513430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-dfab5ad3-1a5b-4757-ac1e-ed9e73565904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:54.554005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:54.623557 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 700/1405] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:54.671534 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7172a13-eea0-4b83-b49d-5315f045039f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:54.673095 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b7172a13-eea0-4b83-b49d-5315f045039f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:54.840042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7172a13-eea0-4b83-b49d-5315f045039f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:54.885960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:54.977873 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38ceac-7891-4fe0-ab43-335d51d6db54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:54.978486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38ceac-7891-4fe0-ab43-335d51d6db54 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:54.978760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38ceac-7891-4fe0-ab43-335d51d6db54 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:54.979034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38ceac-7891-4fe0-ab43-335d51d6db54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:54.980780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-df38ceac-7891-4fe0-ab43-335d51d6db54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:55.083375 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38ceac-7891-4fe0-ab43-335d51d6db54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:55.089166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38ceac-7891-4fe0-ab43-335d51d6db54 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:55.110634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 706/1406] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:30:57.419607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53744b-7053-4e12-82ac-bff089f69387 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:57.420232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53744b-7053-4e12-82ac-bff089f69387 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:57.420232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53744b-7053-4e12-82ac-bff089f69387 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:57.421529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 701/1407] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:30:57.426664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61b1b-098a-4df4-bed2-fa2f9548abca None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:57.427095 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61b1b-098a-4df4-bed2-fa2f9548abca None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:57.427196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61b1b-098a-4df4-bed2-fa2f9548abca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:57.491669 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:57.500386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-52e61b1b-098a-4df4-bed2-fa2f9548abca None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:30:57.508037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-52e61b1b-098a-4df4-bed2-fa2f9548abca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:57.614861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 707/1408] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:30:57.666565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53744b-7053-4e12-82ac-bff089f69387 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:30:57.668654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3a53744b-7053-4e12-82ac-bff089f69387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:57.830532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53744b-7053-4e12-82ac-bff089f69387 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:30:57.853743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:30:57.969448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e9c72-c6c6-4b0f-b721-5858c85e3685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:30:57.970060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e9c72-c6c6-4b0f-b721-5858c85e3685 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:30:57.970339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e9c72-c6c6-4b0f-b721-5858c85e3685 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:30:57.970612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e9c72-c6c6-4b0f-b721-5858c85e3685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:30:57.972262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-346e9c72-c6c6-4b0f-b721-5858c85e3685 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:30:58.082999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e9c72-c6c6-4b0f-b721-5858c85e3685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:30:58.084644 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e9c72-c6c6-4b0f-b721-5858c85e3685 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:30:58.104402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 702/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:07.613044 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=44 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:08.102073 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:15.046521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf20cb-7bb1-4b28-ad5b-085f8009d0c6 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:15.047084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf20cb-7bb1-4b28-ad5b-085f8009d0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:15.047084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf20cb-7bb1-4b28-ad5b-085f8009d0c6 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:15.048451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 708/1410] 10.222.0.30 () {54 vars in 918 bytes} [Sun Aug 7 22:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:15.053449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c805ba-29ec-4195-a11e-09f6fba8e424 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:15.053897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c805ba-29ec-4195-a11e-09f6fba8e424 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:15.054009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c805ba-29ec-4195-a11e-09f6fba8e424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:15.129831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d6c805ba-29ec-4195-a11e-09f6fba8e424 None None] MFA Rules not processed for user `db1d7e39e8c14f5db40efa743fc3b652`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:15.131059 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d6c805ba-29ec-4195-a11e-09f6fba8e424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:15.238403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 703/1411] 10.222.0.30 () {58 vars in 1015 bytes} [Sun Aug 7 22:31:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:31:15.252632 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c78933-5c56-460a-8d08-b65a3ef9e3b5 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:15.252912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c78933-5c56-460a-8d08-b65a3ef9e3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:15.253184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c78933-5c56-460a-8d08-b65a3ef9e3b5 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:15.254524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 709/1412] 10.222.0.30 () {54 vars in 918 bytes} [Sun Aug 7 22:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:15.258930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9be18-2ed6-47d8-a436-87be9530ccc6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:15.259213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9be18-2ed6-47d8-a436-87be9530ccc6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:15.259485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9be18-2ed6-47d8-a436-87be9530ccc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:15.330914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:15.333378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-33c9be18-2ed6-47d8-a436-87be9530ccc6 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:15.334658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-33c9be18-2ed6-47d8-a436-87be9530ccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:15.450575 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 704/1413] 10.222.0.30 () {58 vars in 1015 bytes} [Sun Aug 7 22:31:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:17.920595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedabcc8-5f29-4888-a275-c8a09868d29d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:17.921138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedabcc8-5f29-4888-a275-c8a09868d29d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:17.921138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedabcc8-5f29-4888-a275-c8a09868d29d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:17.922429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:17.927729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad31da-5e5f-4540-bcf3-abbda0275803 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:17.928184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad31da-5e5f-4540-bcf3-abbda0275803 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:17.928338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad31da-5e5f-4540-bcf3-abbda0275803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:17.995874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-09ad31da-5e5f-4540-bcf3-abbda0275803 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:17.997136 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-09ad31da-5e5f-4540-bcf3-abbda0275803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:18.051482 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:18.093763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:18.136654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedabcc8-5f29-4888-a275-c8a09868d29d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:18.138163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-eedabcc8-5f29-4888-a275-c8a09868d29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:18.280933 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedabcc8-5f29-4888-a275-c8a09868d29d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:18.353411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:18.393529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176efff-6aad-4f69-a27e-d073cc6cb345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:18.394127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176efff-6aad-4f69-a27e-d073cc6cb345 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:18.394401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176efff-6aad-4f69-a27e-d073cc6cb345 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:18.394671 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176efff-6aad-4f69-a27e-d073cc6cb345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:18.396287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f176efff-6aad-4f69-a27e-d073cc6cb345 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:18.483811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176efff-6aad-4f69-a27e-d073cc6cb345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:18.485438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176efff-6aad-4f69-a27e-d073cc6cb345 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:18.504203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 711/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:19.400649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730e665-61d8-491a-aae7-42b8f5bee5f1 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:19.401235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730e665-61d8-491a-aae7-42b8f5bee5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:19.401235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730e665-61d8-491a-aae7-42b8f5bee5f1 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:19.403166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:31:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:19.408261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0f013-4f00-43ac-be44-d8d0decde95b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:19.408688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0f013-4f00-43ac-be44-d8d0decde95b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:19.408801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0f013-4f00-43ac-be44-d8d0decde95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:19.483546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-76b0f013-4f00-43ac-be44-d8d0decde95b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:19.484801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-76b0f013-4f00-43ac-be44-d8d0decde95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:19.513052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:19.581971 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:19.731138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730e665-61d8-491a-aae7-42b8f5bee5f1 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:19.732597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c730e665-61d8-491a-aae7-42b8f5bee5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:19.877826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730e665-61d8-491a-aae7-42b8f5bee5f1 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:19.918027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:19.990310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02968f99-84d7-454a-8d4c-86d64a7ea00f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:19.990840 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02968f99-84d7-454a-8d4c-86d64a7ea00f service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:19.991119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02968f99-84d7-454a-8d4c-86d64a7ea00f service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:19.991392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02968f99-84d7-454a-8d4c-86d64a7ea00f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:19.993015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-02968f99-84d7-454a-8d4c-86d64a7ea00f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:20.082693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02968f99-84d7-454a-8d4c-86d64a7ea00f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:20.084319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02968f99-84d7-454a-8d4c-86d64a7ea00f service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:20.102865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 707/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:31:20.986399 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f93995-ac6c-49f4-a502-4cd7aa77d88f None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:20.986921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f93995-ac6c-49f4-a502-4cd7aa77d88f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:20.986921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f93995-ac6c-49f4-a502-4cd7aa77d88f None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:20.988330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:20.993235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a5999-b3c6-4746-a560-cbfa7a5052d6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:20.993642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a5999-b3c6-4746-a560-cbfa7a5052d6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:20.993790 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a5999-b3c6-4746-a560-cbfa7a5052d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:21.049566 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:21.071064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-840a5999-b3c6-4746-a560-cbfa7a5052d6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:21.072291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-840a5999-b3c6-4746-a560-cbfa7a5052d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:21.169391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:21.322562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f93995-ac6c-49f4-a502-4cd7aa77d88f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:21.324711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-33f93995-ac6c-49f4-a502-4cd7aa77d88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:21.470816 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f93995-ac6c-49f4-a502-4cd7aa77d88f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:21.483279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:21.583517 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772fca5-0999-4f19-b1fe-c53f39222cd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:21.584054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772fca5-0999-4f19-b1fe-c53f39222cd3 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:21.584334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772fca5-0999-4f19-b1fe-c53f39222cd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:21.584603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772fca5-0999-4f19-b1fe-c53f39222cd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:21.586185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a772fca5-0999-4f19-b1fe-c53f39222cd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:21.672476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772fca5-0999-4f19-b1fe-c53f39222cd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:21.674045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772fca5-0999-4f19-b1fe-c53f39222cd3 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:21.691345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 714/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:22.567495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95b946-9b16-435b-94c3-4b02f01dde14 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:22.568083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95b946-9b16-435b-94c3-4b02f01dde14 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:22.568213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95b946-9b16-435b-94c3-4b02f01dde14 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:22.569569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:22.575228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa202-c1d2-4dad-9652-4656cde9e1f7 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:22.575639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa202-c1d2-4dad-9652-4656cde9e1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:22.575775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa202-c1d2-4dad-9652-4656cde9e1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:22.595278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:22.651290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-85baa202-c1d2-4dad-9652-4656cde9e1f7 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:22.652633 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-85baa202-c1d2-4dad-9652-4656cde9e1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:22.747568 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:31:22.898864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95b946-9b16-435b-94c3-4b02f01dde14 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:22.900322 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5d95b946-9b16-435b-94c3-4b02f01dde14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:23.009526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:23.045329 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95b946-9b16-435b-94c3-4b02f01dde14 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:23.155927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319f838-f9e0-4c06-abb0-4aa83e7771e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:23.156471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319f838-f9e0-4c06-abb0-4aa83e7771e7 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:23.156744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319f838-f9e0-4c06-abb0-4aa83e7771e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:23.157012 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319f838-f9e0-4c06-abb0-4aa83e7771e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:23.158598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9319f838-f9e0-4c06-abb0-4aa83e7771e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:23.233197 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:23.246495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9319f838-f9e0-4c06-abb0-4aa83e7771e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:23.248119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9319f838-f9e0-4c06-abb0-4aa83e7771e7 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:23.266613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 710/1425] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:24.128837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fb48a-ee6e-4869-af40-883987871366 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:24.129355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fb48a-ee6e-4869-af40-883987871366 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:24.129355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fb48a-ee6e-4869-af40-883987871366 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:24.130695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:24.136300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5e4eb-7771-4591-a200-5312e63625be None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:24.136836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5e4eb-7771-4591-a200-5312e63625be None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:24.137120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5e4eb-7771-4591-a200-5312e63625be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:24.213388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ede5e4eb-7771-4591-a200-5312e63625be None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:24.214741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ede5e4eb-7771-4591-a200-5312e63625be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:24.309375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:24.460470 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fb48a-ee6e-4869-af40-883987871366 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:24.461923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bb0fb48a-ee6e-4869-af40-883987871366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:24.551686 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:24.611829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fb48a-ee6e-4869-af40-883987871366 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:24.723939 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cff9b-91e4-4923-94e4-9a6342e8d0c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:24.724495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cff9b-91e4-4923-94e4-9a6342e8d0c3 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:24.724774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cff9b-91e4-4923-94e4-9a6342e8d0c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:24.725027 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cff9b-91e4-4923-94e4-9a6342e8d0c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:24.726608 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b89cff9b-91e4-4923-94e4-9a6342e8d0c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:24.774114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:24.815871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cff9b-91e4-4923-94e4-9a6342e8d0c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:24.817500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cff9b-91e4-4923-94e4-9a6342e8d0c3 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:24.835152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 717/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:25.712049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bccdb-3cd9-42a8-9d04-f5cc1a9af166 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:25.712606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bccdb-3cd9-42a8-9d04-f5cc1a9af166 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:25.712606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bccdb-3cd9-42a8-9d04-f5cc1a9af166 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:25.714110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:25.718962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643821d-5be4-4a8d-bc08-afd2f8474d9e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:25.719377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643821d-5be4-4a8d-bc08-afd2f8474d9e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:25.719515 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643821d-5be4-4a8d-bc08-afd2f8474d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:25.795160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f643821d-5be4-4a8d-bc08-afd2f8474d9e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:25.796389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f643821d-5be4-4a8d-bc08-afd2f8474d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:25.893065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:26.044480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bccdb-3cd9-42a8-9d04-f5cc1a9af166 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:26.046070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4a2bccdb-3cd9-42a8-9d04-f5cc1a9af166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:26.095130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:26.193047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bccdb-3cd9-42a8-9d04-f5cc1a9af166 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:26.303735 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec0a1a-9c24-48e6-8414-b41456b63527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:26.304367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec0a1a-9c24-48e6-8414-b41456b63527 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:26.304640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec0a1a-9c24-48e6-8414-b41456b63527 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:26.304909 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec0a1a-9c24-48e6-8414-b41456b63527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:26.306487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-75ec0a1a-9c24-48e6-8414-b41456b63527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:26.313339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:26.397009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec0a1a-9c24-48e6-8414-b41456b63527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:26.398697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec0a1a-9c24-48e6-8414-b41456b63527 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:26.417447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 713/1431] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:27.256261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e177eb-c45c-4c7c-af85-b2a1ff290357 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:27.256790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e177eb-c45c-4c7c-af85-b2a1ff290357 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:27.256790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e177eb-c45c-4c7c-af85-b2a1ff290357 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:27.258128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:27.263716 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71b841-0b6f-408e-81e4-fb7298fa71b7 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:27.264173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71b841-0b6f-408e-81e4-fb7298fa71b7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:27.264294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71b841-0b6f-408e-81e4-fb7298fa71b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:27.339345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ce71b841-0b6f-408e-81e4-fb7298fa71b7 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:27.340588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ce71b841-0b6f-408e-81e4-fb7298fa71b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:27.407699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:27.436932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:27.585118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e177eb-c45c-4c7c-af85-b2a1ff290357 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:27.586628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-86e177eb-c45c-4c7c-af85-b2a1ff290357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:27.603755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:27.728444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e177eb-c45c-4c7c-af85-b2a1ff290357 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:27.815037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:27.842756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5ef16-74c3-4126-af05-5ec3a409197f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:27.843352 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5ef16-74c3-4126-af05-5ec3a409197f service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:27.843558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5ef16-74c3-4126-af05-5ec3a409197f service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:27.843823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5ef16-74c3-4126-af05-5ec3a409197f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:27.845657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-49c5ef16-74c3-4126-af05-5ec3a409197f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:27.933511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5ef16-74c3-4126-af05-5ec3a409197f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:27.935121 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5ef16-74c3-4126-af05-5ec3a409197f service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:27.954286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 720/1434] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:28.810020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e364b-42c0-4484-9695-d420e7c32820 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:28.810637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e364b-42c0-4484-9695-d420e7c32820 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:28.810637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e364b-42c0-4484-9695-d420e7c32820 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:28.811946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:31:28.816719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008ff9-38f9-4989-8dc2-ba7123d1d71b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:28.817140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008ff9-38f9-4989-8dc2-ba7123d1d71b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:28.817271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008ff9-38f9-4989-8dc2-ba7123d1d71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:28.892387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e8008ff9-38f9-4989-8dc2-ba7123d1d71b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:28.893590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e8008ff9-38f9-4989-8dc2-ba7123d1d71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:28.934537 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:28.987944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:29.137128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e364b-42c0-4484-9695-d420e7c32820 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:29.138560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f86e364b-42c0-4484-9695-d420e7c32820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:29.283940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e364b-42c0-4484-9695-d420e7c32820 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:29.337779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:29.397825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e60bd-e089-45e6-be7c-b6446f6a85a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:29.398311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e60bd-e089-45e6-be7c-b6446f6a85a6 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:29.398582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e60bd-e089-45e6-be7c-b6446f6a85a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:29.398848 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e60bd-e089-45e6-be7c-b6446f6a85a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:29.400471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bb3e60bd-e089-45e6-be7c-b6446f6a85a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:29.492369 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e60bd-e089-45e6-be7c-b6446f6a85a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:29.493933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e60bd-e089-45e6-be7c-b6446f6a85a6 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:29.511955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 716/1437] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:30.376878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe06f2-6872-4fc4-b884-591296ab7ea1 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:30.377723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe06f2-6872-4fc4-b884-591296ab7ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:30.377723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe06f2-6872-4fc4-b884-591296ab7ea1 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:30.379033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:30.383995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca939a-ddde-4809-ad2a-14505b5e02c3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:30.384467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca939a-ddde-4809-ad2a-14505b5e02c3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:30.384576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca939a-ddde-4809-ad2a-14505b5e02c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:30.460455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-32ca939a-ddde-4809-ad2a-14505b5e02c3 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:30.461660 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-32ca939a-ddde-4809-ad2a-14505b5e02c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:30.463779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:30.559362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:30.710462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe06f2-6872-4fc4-b884-591296ab7ea1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:30.711981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a0fe06f2-6872-4fc4-b884-591296ab7ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:30.856143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe06f2-6872-4fc4-b884-591296ab7ea1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:30.883411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:30.967951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688db5c1-43b9-492b-b2e0-782ace7382fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:30.968516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db5c1-43b9-492b-b2e0-782ace7382fc service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:30.968792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db5c1-43b9-492b-b2e0-782ace7382fc service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:30.969063 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db5c1-43b9-492b-b2e0-782ace7382fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:30.970657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-688db5c1-43b9-492b-b2e0-782ace7382fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:31.059728 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688db5c1-43b9-492b-b2e0-782ace7382fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:31.061348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688db5c1-43b9-492b-b2e0-782ace7382fc service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:31.079410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 723/1440] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:32.989450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f71ba0-0ec2-4bfa-88c2-1ff0aa70e09a None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:32.989993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f71ba0-0ec2-4bfa-88c2-1ff0aa70e09a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:32.989993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f71ba0-0ec2-4bfa-88c2-1ff0aa70e09a None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:32.991353 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:32.997019 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0372e-5589-4a0b-b6dc-e027f36cf345 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:32.997418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0372e-5589-4a0b-b6dc-e027f36cf345 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:32.997567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0372e-5589-4a0b-b6dc-e027f36cf345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:33.039076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:33.063900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c2b0372e-5589-4a0b-b6dc-e027f36cf345 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:33.065152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c2b0372e-5589-4a0b-b6dc-e027f36cf345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:33.160750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:33.200652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f71ba0-0ec2-4bfa-88c2-1ff0aa70e09a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:33.202193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-78f71ba0-0ec2-4bfa-88c2-1ff0aa70e09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:33.303238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f71ba0-0ec2-4bfa-88c2-1ff0aa70e09a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:33.336318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:33.377805 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ddff2-08b0-4aec-b7f1-0688715111ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:33.378416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ddff2-08b0-4aec-b7f1-0688715111ab service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:33.378695 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ddff2-08b0-4aec-b7f1-0688715111ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:33.378965 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ddff2-08b0-4aec-b7f1-0688715111ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:33.380730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f24ddff2-08b0-4aec-b7f1-0688715111ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:33.489647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ddff2-08b0-4aec-b7f1-0688715111ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:33.490988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ddff2-08b0-4aec-b7f1-0688715111ab service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:33.511522 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 719/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:35.653829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7da9a6-7973-4ef8-9709-f468f02fb5d5 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:35.654401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7da9a6-7973-4ef8-9709-f468f02fb5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:35.654401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7da9a6-7973-4ef8-9709-f468f02fb5d5 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:35.655735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:35.661043 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6715a63-4405-432a-87ae-2f65668da896 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:35.661489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6715a63-4405-432a-87ae-2f65668da896 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:35.661593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6715a63-4405-432a-87ae-2f65668da896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:35.726745 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a6715a63-4405-432a-87ae-2f65668da896 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:35.727947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a6715a63-4405-432a-87ae-2f65668da896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:35.740524 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:35.822254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:35.861003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7da9a6-7973-4ef8-9709-f468f02fb5d5 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:35.862574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bc7da9a6-7973-4ef8-9709-f468f02fb5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:35.966926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7da9a6-7973-4ef8-9709-f468f02fb5d5 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:35.980150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:36.043264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db2647-dbc3-4dfb-a933-118f704d6870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:36.043859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db2647-dbc3-4dfb-a933-118f704d6870 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:36.044170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db2647-dbc3-4dfb-a933-118f704d6870 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:36.044441 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db2647-dbc3-4dfb-a933-118f704d6870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:36.046109 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d3db2647-dbc3-4dfb-a933-118f704d6870 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:36.155380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db2647-dbc3-4dfb-a933-118f704d6870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:36.156859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db2647-dbc3-4dfb-a933-118f704d6870 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:36.175042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 726/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:38.524454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2914e3a-28d5-4fac-9280-f6870e3054ed None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:38.525046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2914e3a-28d5-4fac-9280-f6870e3054ed None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:38.525046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2914e3a-28d5-4fac-9280-f6870e3054ed None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:38.526833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:38.531978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3e0df-d617-4c95-afd7-f874f6545206 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:38.532497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3e0df-d617-4c95-afd7-f874f6545206 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:38.532600 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3e0df-d617-4c95-afd7-f874f6545206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:38.579830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:38.597710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3cc3e0df-d617-4c95-afd7-f874f6545206 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:38.602818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3cc3e0df-d617-4c95-afd7-f874f6545206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:38.695529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:38.735698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2914e3a-28d5-4fac-9280-f6870e3054ed None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:38.737261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a2914e3a-28d5-4fac-9280-f6870e3054ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:38.840462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2914e3a-28d5-4fac-9280-f6870e3054ed None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:38.880385 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:38.913558 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe21737-3caa-441b-b7d0-0c515c20bf81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:38.914146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe21737-3caa-441b-b7d0-0c515c20bf81 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:38.914431 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe21737-3caa-441b-b7d0-0c515c20bf81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:38.914704 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe21737-3caa-441b-b7d0-0c515c20bf81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:38.918635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-dbe21737-3caa-441b-b7d0-0c515c20bf81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:39.026597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe21737-3caa-441b-b7d0-0c515c20bf81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:39.028048 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe21737-3caa-441b-b7d0-0c515c20bf81 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:39.046628 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 722/1449] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:41.177140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e15c0-6099-415c-95ca-875847c2f8b7 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:41.177699 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e15c0-6099-415c-95ca-875847c2f8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:41.177699 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e15c0-6099-415c-95ca-875847c2f8b7 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:41.179064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:41.184385 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e52d55-66c1-428d-9fa7-a5d1fc696310 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:41.184791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e52d55-66c1-428d-9fa7-a5d1fc696310 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:41.184936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e52d55-66c1-428d-9fa7-a5d1fc696310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:41.251890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-93e52d55-66c1-428d-9fa7-a5d1fc696310 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:41.253333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-93e52d55-66c1-428d-9fa7-a5d1fc696310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:41.274161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:41.349594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:41.389838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e15c0-6099-415c-95ca-875847c2f8b7 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:41.391425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fb3e15c0-6099-415c-95ca-875847c2f8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:41.493603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e15c0-6099-415c-95ca-875847c2f8b7 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:41.514425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:41.567841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e6cdd-382c-444b-bc1a-fb867d931585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:41.568471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e6cdd-382c-444b-bc1a-fb867d931585 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:41.568745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e6cdd-382c-444b-bc1a-fb867d931585 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:41.569014 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e6cdd-382c-444b-bc1a-fb867d931585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:41.570601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-be7e6cdd-382c-444b-bc1a-fb867d931585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:41.677823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e6cdd-382c-444b-bc1a-fb867d931585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:41.679260 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e6cdd-382c-444b-bc1a-fb867d931585 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:41.698008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 729/1452] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:43.895065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b91170-ec48-4f5e-814b-7e5b5b04a3a0 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:43.895638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b91170-ec48-4f5e-814b-7e5b5b04a3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:43.895638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b91170-ec48-4f5e-814b-7e5b5b04a3a0 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:43.897019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:43.902774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c940f5-9c37-4401-9089-83de3c48bb66 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:43.903225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c940f5-9c37-4401-9089-83de3c48bb66 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:43.903321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c940f5-9c37-4401-9089-83de3c48bb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:43.966680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:43.969582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c8c940f5-9c37-4401-9089-83de3c48bb66 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:43.970807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c8c940f5-9c37-4401-9089-83de3c48bb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:44.066639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:44.105531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b91170-ec48-4f5e-814b-7e5b5b04a3a0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:44.107136 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-26b91170-ec48-4f5e-814b-7e5b5b04a3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:44.211276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b91170-ec48-4f5e-814b-7e5b5b04a3a0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:44.259603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:44.284264 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47152ae2-2a40-4f37-9586-9f5f669cbbf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:44.284862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47152ae2-2a40-4f37-9586-9f5f669cbbf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:44.285138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47152ae2-2a40-4f37-9586-9f5f669cbbf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:44.285409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47152ae2-2a40-4f37-9586-9f5f669cbbf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:44.286990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-47152ae2-2a40-4f37-9586-9f5f669cbbf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:44.395738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47152ae2-2a40-4f37-9586-9f5f669cbbf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:44.397186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47152ae2-2a40-4f37-9586-9f5f669cbbf1 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:44.415488 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 725/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:46.465578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a2c34-e502-4f84-9520-256a5a6b5a1f None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:46.466261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a2c34-e502-4f84-9520-256a5a6b5a1f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:46.466261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a2c34-e502-4f84-9520-256a5a6b5a1f None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:46.467477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:46.472723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6d00a-d4bd-4a7f-b1cb-53d69da32eb1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:46.473167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6d00a-d4bd-4a7f-b1cb-53d69da32eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:46.473281 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6d00a-d4bd-4a7f-b1cb-53d69da32eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:46.541494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d7c6d00a-d4bd-4a7f-b1cb-53d69da32eb1 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:46.542757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d7c6d00a-d4bd-4a7f-b1cb-53d69da32eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:46.570969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:46.638850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:46.677047 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a2c34-e502-4f84-9520-256a5a6b5a1f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:46.678571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fe9a2c34-e502-4f84-9520-256a5a6b5a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:46.779571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a2c34-e502-4f84-9520-256a5a6b5a1f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:46.808102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:46.852298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7690105-95ad-48f9-8522-b7d3d75248ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:46.852899 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7690105-95ad-48f9-8522-b7d3d75248ee service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:46.853178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7690105-95ad-48f9-8522-b7d3d75248ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:46.854042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7690105-95ad-48f9-8522-b7d3d75248ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:46.855637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f7690105-95ad-48f9-8522-b7d3d75248ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:46.973009 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7690105-95ad-48f9-8522-b7d3d75248ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:46.974468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7690105-95ad-48f9-8522-b7d3d75248ee service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:46.994165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 732/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:49.142085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee8ed9-25a6-4f5c-93e0-0a627e8f2c2d None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:49.142639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee8ed9-25a6-4f5c-93e0-0a627e8f2c2d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:49.142639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee8ed9-25a6-4f5c-93e0-0a627e8f2c2d None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:49.143995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:49.149291 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ed710-0178-4492-aa80-dd4ea1afe21d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:49.149715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ed710-0178-4492-aa80-dd4ea1afe21d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:49.149842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ed710-0178-4492-aa80-dd4ea1afe21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:49.215027 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8b5ed710-0178-4492-aa80-dd4ea1afe21d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:49.216171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8b5ed710-0178-4492-aa80-dd4ea1afe21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:49.224374 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:49.312526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:49.351198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee8ed9-25a6-4f5c-93e0-0a627e8f2c2d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:49.352764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-81ee8ed9-25a6-4f5c-93e0-0a627e8f2c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:49.457001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee8ed9-25a6-4f5c-93e0-0a627e8f2c2d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:49.513023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:49.532030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6322c9f-3129-433d-876f-53167fad32f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:49.532640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6322c9f-3129-433d-876f-53167fad32f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:49.532919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6322c9f-3129-433d-876f-53167fad32f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:49.533188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6322c9f-3129-433d-876f-53167fad32f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:49.534832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e6322c9f-3129-433d-876f-53167fad32f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:49.642424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6322c9f-3129-433d-876f-53167fad32f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:49.643843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6322c9f-3129-433d-876f-53167fad32f4 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:49.662494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 728/1461] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:31:51.713649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab34589-9cc8-4e86-9847-006a0db9b9e9 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:51.714249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab34589-9cc8-4e86-9847-006a0db9b9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:51.714249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab34589-9cc8-4e86-9847-006a0db9b9e9 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:51.715637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:51.720780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229c0af-00bb-48ac-ba5d-e95fa2b59b74 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:51.721234 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229c0af-00bb-48ac-ba5d-e95fa2b59b74 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:51.721339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229c0af-00bb-48ac-ba5d-e95fa2b59b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:51.788362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8229c0af-00bb-48ac-ba5d-e95fa2b59b74 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:51.789591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8229c0af-00bb-48ac-ba5d-e95fa2b59b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:51.829219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:51.887293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:51.925798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab34589-9cc8-4e86-9847-006a0db9b9e9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:51.927334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6ab34589-9cc8-4e86-9847-006a0db9b9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:52.030981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab34589-9cc8-4e86-9847-006a0db9b9e9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:52.069999 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:52.106049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729655a-ae52-41e7-b083-7257f7422b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:52.106751 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729655a-ae52-41e7-b083-7257f7422b74 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:52.107127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729655a-ae52-41e7-b083-7257f7422b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:52.107501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729655a-ae52-41e7-b083-7257f7422b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:52.109509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3729655a-ae52-41e7-b083-7257f7422b74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:52.217466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729655a-ae52-41e7-b083-7257f7422b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:52.218882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729655a-ae52-41e7-b083-7257f7422b74 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:52.236904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 735/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:54.262211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ba328-7aeb-4606-908f-a04b1b4b5145 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:54.262754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ba328-7aeb-4606-908f-a04b1b4b5145 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:54.262754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ba328-7aeb-4606-908f-a04b1b4b5145 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:54.264115 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:31:54.269432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cc904-7050-4b68-b42d-bb1ff4987512 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:54.269851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cc904-7050-4b68-b42d-bb1ff4987512 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:54.269988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cc904-7050-4b68-b42d-bb1ff4987512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:54.336538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-641cc904-7050-4b68-b42d-bb1ff4987512 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:54.337860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-641cc904-7050-4b68-b42d-bb1ff4987512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:54.355504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:54.436977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:54.473460 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ba328-7aeb-4606-908f-a04b1b4b5145 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:54.474982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-583ba328-7aeb-4606-908f-a04b1b4b5145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:54.580823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ba328-7aeb-4606-908f-a04b1b4b5145 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:54.646762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:54.653466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e831e07-fce3-4777-9aad-dde9bf0a41b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:54.654065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e831e07-fce3-4777-9aad-dde9bf0a41b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:54.654341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e831e07-fce3-4777-9aad-dde9bf0a41b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:54.654610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e831e07-fce3-4777-9aad-dde9bf0a41b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:54.656257 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6e831e07-fce3-4777-9aad-dde9bf0a41b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:54.763217 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e831e07-fce3-4777-9aad-dde9bf0a41b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:54.764669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e831e07-fce3-4777-9aad-dde9bf0a41b4 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:54.783114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 731/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:31:56.783558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8cc5d-529a-4eb0-be65-affa08d95eb1 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:56.784138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8cc5d-529a-4eb0-be65-affa08d95eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:56.784138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8cc5d-529a-4eb0-be65-affa08d95eb1 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:56.785486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:31:56.790602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377d1ac-1538-4e4e-a2e5-892ef7574bdb None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:56.791044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377d1ac-1538-4e4e-a2e5-892ef7574bdb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:56.791151 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377d1ac-1538-4e4e-a2e5-892ef7574bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:56.856102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8377d1ac-1538-4e4e-a2e5-892ef7574bdb None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:31:56.857351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8377d1ac-1538-4e4e-a2e5-892ef7574bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:56.906613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:56.955923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:31:56.992568 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8cc5d-529a-4eb0-be65-affa08d95eb1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:31:56.994115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-24a8cc5d-529a-4eb0-be65-affa08d95eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:57.099012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8cc5d-529a-4eb0-be65-affa08d95eb1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:31:57.145967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:31:57.172157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7c25f-4017-4009-982e-f3ab770e80a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:31:57.172766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c25f-4017-4009-982e-f3ab770e80a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:31:57.173045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c25f-4017-4009-982e-f3ab770e80a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:31:57.173345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c25f-4017-4009-982e-f3ab770e80a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:31:57.175078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-01d7c25f-4017-4009-982e-f3ab770e80a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:31:57.283547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7c25f-4017-4009-982e-f3ab770e80a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:31:57.285050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7c25f-4017-4009-982e-f3ab770e80a6 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:31:57.304416 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 738/1470] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:32:06.954124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=22 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:07.302084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=66 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:16.890934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddc0c9-cb92-4f24-90e5-6f34bec81df4 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:16.891494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddc0c9-cb92-4f24-90e5-6f34bec81df4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:16.891494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddc0c9-cb92-4f24-90e5-6f34bec81df4 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:16.892841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 733/1471] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:32:16.898202 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ef8c-eaab-4653-8227-589b9a809411 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:16.898673 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ef8c-eaab-4653-8227-589b9a809411 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:16.898771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ef8c-eaab-4653-8227-589b9a809411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:16.967792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-76a1ef8c-eaab-4653-8227-589b9a809411 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:32:16.969131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-76a1ef8c-eaab-4653-8227-589b9a809411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:17.075822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 739/1472] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:32:17.159095 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5705-5455-41ef-b798-aa1889086c40 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:17.159379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5705-5455-41ef-b798-aa1889086c40 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:17.159657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5705-5455-41ef-b798-aa1889086c40 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:17.161007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 734/1473] 10.222.0.30 () {54 vars in 966 bytes} [Sun Aug 7 22:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:32:17.165872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f6e89-7f9c-4b50-bdb6-d909e52d65df None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:17.166784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f6e89-7f9c-4b50-bdb6-d909e52d65df None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:17.167065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f6e89-7f9c-4b50-bdb6-d909e52d65df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:17.245315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0e4f6e89-7f9c-4b50-bdb6-d909e52d65df None None] MFA Rules not processed for user `cfb2129c006f4ddeba7d128a8d43d5de`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:32:17.246547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0e4f6e89-7f9c-4b50-bdb6-d909e52d65df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:17.267146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:17.323483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 740/1474] 10.222.0.30 () {58 vars in 1063 bytes} [Sun Aug 7 22:32:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:32:17.331138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa5705-5455-41ef-b798-aa1889086c40 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:32:17.332639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ccfa5705-5455-41ef-b798-aa1889086c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:17.434674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa5705-5455-41ef-b798-aa1889086c40 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:32:17.506342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dc00b-e4e8-44e2-b581-89aabe9ad039 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb2129c006f4ddeba7d128a8d43d5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:32:17.506936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dc00b-e4e8-44e2-b581-89aabe9ad039 service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:17.507210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dc00b-e4e8-44e2-b581-89aabe9ad039 service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:17.507475 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dc00b-e4e8-44e2-b581-89aabe9ad039 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:17.509089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3b8dc00b-e4e8-44e2-b581-89aabe9ad039 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:17.562249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:17.616142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dc00b-e4e8-44e2-b581-89aabe9ad039 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:32:17.617573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dc00b-e4e8-44e2-b581-89aabe9ad039 service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:32:17.636490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 735/1475] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:32:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:32:19.720272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb512ae6-23f2-4a89-8bce-da4357fff866 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:19.720841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb512ae6-23f2-4a89-8bce-da4357fff866 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:19.720841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb512ae6-23f2-4a89-8bce-da4357fff866 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:19.722220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 741/1476] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:32:19.727622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8bf3c-7769-42e4-8e87-ad933470e5aa None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:19.728059 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8bf3c-7769-42e4-8e87-ad933470e5aa None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:19.728238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8bf3c-7769-42e4-8e87-ad933470e5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:19.794600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-41e8bf3c-7769-42e4-8e87-ad933470e5aa None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:32:19.795808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-41e8bf3c-7769-42e4-8e87-ad933470e5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:19.892031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 736/1477] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:32:19.916430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb512ae6-23f2-4a89-8bce-da4357fff866 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:32:19.918004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bb512ae6-23f2-4a89-8bce-da4357fff866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:20.048047 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb512ae6-23f2-4a89-8bce-da4357fff866 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:32:20.096893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:20.141237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5b170-b734-45d9-8a65-a409aa79f709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:32:20.141839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5b170-b734-45d9-8a65-a409aa79f709 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:20.142110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5b170-b734-45d9-8a65-a409aa79f709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:20.142372 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5b170-b734-45d9-8a65-a409aa79f709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:20.143004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5b170-b734-45d9-8a65-a409aa79f709 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:32:20.144753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-cea5b170-b734-45d9-8a65-a409aa79f709 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:32:20.144753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Aug 07 22:32:20.144753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: Aug 07 22:32:20.145887 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5b170-b734-45d9-8a65-a409aa79f709 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:32:20.164297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 742/1478] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 22:32:19 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:32:22.181234 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafe110-db51-4e5f-b50c-f82408d0c919 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:22.181998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafe110-db51-4e5f-b50c-f82408d0c919 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:22.181998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafe110-db51-4e5f-b50c-f82408d0c919 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:22.183201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 737/1479] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:32:22.188281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517221a-848d-4d92-a12b-4a5952e3aef6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:22.188711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517221a-848d-4d92-a12b-4a5952e3aef6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:22.188825 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517221a-848d-4d92-a12b-4a5952e3aef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:22.254514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6517221a-848d-4d92-a12b-4a5952e3aef6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:32:22.255663 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6517221a-848d-4d92-a12b-4a5952e3aef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:22.353042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 743/1480] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:32:22.397120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafe110-db51-4e5f-b50c-f82408d0c919 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:32:22.398638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9aafe110-db51-4e5f-b50c-f82408d0c919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:22.409177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:22.544406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafe110-db51-4e5f-b50c-f82408d0c919 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:32:22.622577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:22.657310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12947b-8e2d-4265-9ec0-199f8cea6b54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:32:22.657903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12947b-8e2d-4265-9ec0-199f8cea6b54 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:22.658180 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12947b-8e2d-4265-9ec0-199f8cea6b54 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:22.658451 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12947b-8e2d-4265-9ec0-199f8cea6b54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:22.660057 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7a12947b-8e2d-4265-9ec0-199f8cea6b54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:22.747287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12947b-8e2d-4265-9ec0-199f8cea6b54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:32:22.748882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12947b-8e2d-4265-9ec0-199f8cea6b54 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:32:22.766965 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 738/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:32:24.951622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b20209-2809-472d-ab70-9ddb917bd740 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:24.952078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b20209-2809-472d-ab70-9ddb917bd740 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:24.952194 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b20209-2809-472d-ab70-9ddb917bd740 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:24.953569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 744/1482] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:32:24.958695 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2449b10-2f1d-4c4b-9a45-cb53e38153cc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:24.959131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2449b10-2f1d-4c4b-9a45-cb53e38153cc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:24.959279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2449b10-2f1d-4c4b-9a45-cb53e38153cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:25.026188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d2449b10-2f1d-4c4b-9a45-cb53e38153cc None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:32:25.027395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d2449b10-2f1d-4c4b-9a45-cb53e38153cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:25.073977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:25.124304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 739/1483] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:32:25.160346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b20209-2809-472d-ab70-9ddb917bd740 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:32:25.161878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-78b20209-2809-472d-ab70-9ddb917bd740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:25.186563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:25.266070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b20209-2809-472d-ab70-9ddb917bd740 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:32:25.340651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1db551-18ff-4fd4-9f5c-2f395ea4e967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:32:25.341259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1db551-18ff-4fd4-9f5c-2f395ea4e967 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:25.341539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1db551-18ff-4fd4-9f5c-2f395ea4e967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:25.341809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1db551-18ff-4fd4-9f5c-2f395ea4e967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:25.343408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ad1db551-18ff-4fd4-9f5c-2f395ea4e967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:25.418043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:25.453434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1db551-18ff-4fd4-9f5c-2f395ea4e967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:32:25.454891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1db551-18ff-4fd4-9f5c-2f395ea4e967 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:32:25.473922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 745/1484] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:32:27.520570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b6324-9bc4-4f0b-9497-f4ffa9d62892 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:27.521130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b6324-9bc4-4f0b-9497-f4ffa9d62892 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:27.521325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b6324-9bc4-4f0b-9497-f4ffa9d62892 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:27.523781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 740/1485] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:32:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:32:27.529801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c114ae-86ab-4a8d-a402-f4a9b673a0a2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:27.530254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c114ae-86ab-4a8d-a402-f4a9b673a0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:27.530377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c114ae-86ab-4a8d-a402-f4a9b673a0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:27.594897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e4c114ae-86ab-4a8d-a402-f4a9b673a0a2 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:32:27.596130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e4c114ae-86ab-4a8d-a402-f4a9b673a0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:27.690409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 746/1486] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:32:27.729220 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b6324-9bc4-4f0b-9497-f4ffa9d62892 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:32:27.730732 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a63b6324-9bc4-4f0b-9497-f4ffa9d62892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:27.832776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b6324-9bc4-4f0b-9497-f4ffa9d62892 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:32:27.903006 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:27.904826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c578c-a63b-4936-8e81-9d13dd9a78f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:32:27.905418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c578c-a63b-4936-8e81-9d13dd9a78f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:27.905693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c578c-a63b-4936-8e81-9d13dd9a78f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:27.905963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c578c-a63b-4936-8e81-9d13dd9a78f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:27.907533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-510c578c-a63b-4936-8e81-9d13dd9a78f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:28.016419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c578c-a63b-4936-8e81-9d13dd9a78f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:32:28.017856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c578c-a63b-4936-8e81-9d13dd9a78f2 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:32:28.036614 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 741/1487] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:32:30.140421 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49696b0-e667-4ae7-a939-e3b9f1471e4b None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:30.141014 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49696b0-e667-4ae7-a939-e3b9f1471e4b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:30.141014 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49696b0-e667-4ae7-a939-e3b9f1471e4b None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:30.142381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 747/1488] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:32:30.147602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bad96d-0169-43a2-badb-db300477a6a3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:30.148042 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bad96d-0169-43a2-badb-db300477a6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:30.148202 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bad96d-0169-43a2-badb-db300477a6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:30.214830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-74bad96d-0169-43a2-badb-db300477a6a3 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:32:30.215959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-74bad96d-0169-43a2-badb-db300477a6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:30.268109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:30.312337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 742/1489] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:32:30.354533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49696b0-e667-4ae7-a939-e3b9f1471e4b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:32:30.356263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d49696b0-e667-4ae7-a939-e3b9f1471e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:30.393009 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:30.461411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49696b0-e667-4ae7-a939-e3b9f1471e4b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:32:30.535077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33757d71-f8b7-46b5-a28f-ec466abba16a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:32:30.535693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33757d71-f8b7-46b5-a28f-ec466abba16a service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:30.535974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33757d71-f8b7-46b5-a28f-ec466abba16a service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:30.536279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33757d71-f8b7-46b5-a28f-ec466abba16a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:30.537928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-33757d71-f8b7-46b5-a28f-ec466abba16a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:30.617514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:30.646702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33757d71-f8b7-46b5-a28f-ec466abba16a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:32:30.648157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33757d71-f8b7-46b5-a28f-ec466abba16a service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:32:30.666919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 748/1490] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:32:33.050229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2340b1-5bc3-4d8d-b47c-b87475e9c3ef None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:33.050664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2340b1-5bc3-4d8d-b47c-b87475e9c3ef None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:33.050780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2340b1-5bc3-4d8d-b47c-b87475e9c3ef None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:33.052194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 743/1491] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 22:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:32:33.058133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ddae-141f-4e31-a48b-0f32bdceba2c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:33.058559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ddae-141f-4e31-a48b-0f32bdceba2c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:33.058664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ddae-141f-4e31-a48b-0f32bdceba2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:33.123041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-01d6ddae-141f-4e31-a48b-0f32bdceba2c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:32:33.124291 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-01d6ddae-141f-4e31-a48b-0f32bdceba2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:33.219860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 749/1492] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 22:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:32:33.260744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2340b1-5bc3-4d8d-b47c-b87475e9c3ef None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:32:33.262266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9b2340b1-5bc3-4d8d-b47c-b87475e9c3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:33.365882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2340b1-5bc3-4d8d-b47c-b87475e9c3ef None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:32:33.441643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b1886-19b0-4b48-91c5-1579ecf82607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:32:33.442278 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b1886-19b0-4b48-91c5-1579ecf82607 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:32:33.442558 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b1886-19b0-4b48-91c5-1579ecf82607 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:32:33.442830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b1886-19b0-4b48-91c5-1579ecf82607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:32:33.444475 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-029b1886-19b0-4b48-91c5-1579ecf82607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:32:33.453764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:33.552693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b1886-19b0-4b48-91c5-1579ecf82607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:32:33.554117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b1886-19b0-4b48-91c5-1579ecf82607 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:32:33.573261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 744/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:32:43.218077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=82 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:32:43.570837 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=50 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:33:51.808186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4800f-1b03-4421-b167-9f4370753c05 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:51.808788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4800f-1b03-4421-b167-9f4370753c05 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:51.808788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4800f-1b03-4421-b167-9f4370753c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:51.884530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-35b4800f-1b03-4421-b167-9f4370753c05 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:33:51.885752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-35b4800f-1b03-4421-b167-9f4370753c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:51.984200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 750/1494] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:33:51.992550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742a4d7-6f97-41e9-b42a-783cc5f6cc0f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:51.993165 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742a4d7-6f97-41e9-b42a-783cc5f6cc0f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:51.993291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742a4d7-6f97-41e9-b42a-783cc5f6cc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:52.068812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-5742a4d7-6f97-41e9-b42a-783cc5f6cc0f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:33:52.069956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5742a4d7-6f97-41e9-b42a-783cc5f6cc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:52.166090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 745/1495] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:33:52.170909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4800f-1b03-4421-b167-9f4370753c05 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:33:52.172152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-35b4800f-1b03-4421-b167-9f4370753c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:52.184141 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4800f-1b03-4421-b167-9f4370753c05 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:33:52.244903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:33:52.280661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec972c-67ef-429c-bf15-6a50723a1201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:33:52.281271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec972c-67ef-429c-bf15-6a50723a1201 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:52.281553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec972c-67ef-429c-bf15-6a50723a1201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:52.281830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec972c-67ef-429c-bf15-6a50723a1201 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:52.282476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec972c-67ef-429c-bf15-6a50723a1201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:33:52.282782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec972c-67ef-429c-bf15-6a50723a1201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:33:52.283801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-5eec972c-67ef-429c-bf15-6a50723a1201 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:33:52.283801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Aug 07 22:33:52.283801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: Aug 07 22:33:52.284779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec972c-67ef-429c-bf15-6a50723a1201 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:33:52.296177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 751/1496] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:33:52.304474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742a4d7-6f97-41e9-b42a-783cc5f6cc0f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:33:52.314224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742a4d7-6f97-41e9-b42a-783cc5f6cc0f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:33:52.374959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:33:52.408636 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebdecf-d955-4d9b-8d2a-cd44be105b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:33:52.409241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebdecf-d955-4d9b-8d2a-cd44be105b2d admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:52.409523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebdecf-d955-4d9b-8d2a-cd44be105b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:52.409804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebdecf-d955-4d9b-8d2a-cd44be105b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:52.410448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebdecf-d955-4d9b-8d2a-cd44be105b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:33:52.412239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-06ebdecf-d955-4d9b-8d2a-cd44be105b2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:33:52.412239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Aug 07 22:33:52.412239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: Aug 07 22:33:52.413449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebdecf-d955-4d9b-8d2a-cd44be105b2d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:33:52.528691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 746/1497] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:33:52 2022] POST /identity/v3/projects => generated 377 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:33:52.532890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec972c-67ef-429c-bf15-6a50723a1201 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:33:52.534466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5eec972c-67ef-429c-bf15-6a50723a1201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:52.567655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec972c-67ef-429c-bf15-6a50723a1201 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:33:52.662050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643eb0c5-c17d-45f0-850e-f0a339e602ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:33:52.662571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eb0c5-c17d-45f0-850e-f0a339e602ad admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:52.662848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eb0c5-c17d-45f0-850e-f0a339e602ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:52.663120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eb0c5-c17d-45f0-850e-f0a339e602ad admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:52.663734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643eb0c5-c17d-45f0-850e-f0a339e602ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:33:52.664798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-643eb0c5-c17d-45f0-850e-f0a339e602ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:33:52.664798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Aug 07 22:33:52.664798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: Aug 07 22:33:52.666001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643eb0c5-c17d-45f0-850e-f0a339e602ad admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:33:52.755973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 752/1498] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:33:52 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:33:52.760686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebdecf-d955-4d9b-8d2a-cd44be105b2d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:33:52.762175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-06ebdecf-d955-4d9b-8d2a-cd44be105b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:52.823187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebdecf-d955-4d9b-8d2a-cd44be105b2d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:33:52.905505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:33:52.915931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682577ef-51f4-4bfc-9e88-db7a3868a8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:33:52.916507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682577ef-51f4-4bfc-9e88-db7a3868a8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:52.916787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682577ef-51f4-4bfc-9e88-db7a3868a8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:52.917059 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682577ef-51f4-4bfc-9e88-db7a3868a8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:52.917694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682577ef-51f4-4bfc-9e88-db7a3868a8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:33:52.918696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-682577ef-51f4-4bfc-9e88-db7a3868a8aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:33:52.918696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Aug 07 22:33:52.918696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: Aug 07 22:33:52.919582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682577ef-51f4-4bfc-9e88-db7a3868a8aa admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:33:52.932342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 747/1499] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:33:52.936423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643eb0c5-c17d-45f0-850e-f0a339e602ad admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:33:52.938102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-643eb0c5-c17d-45f0-850e-f0a339e602ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:52.950369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643eb0c5-c17d-45f0-850e-f0a339e602ad admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:33:52.975718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:33:53.047681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5399d-d739-40ce-b056-67cbf1f33d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:33:53.048446 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5399d-d739-40ce-b056-67cbf1f33d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:53.048729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5399d-d739-40ce-b056-67cbf1f33d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:53.049005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5399d-d739-40ce-b056-67cbf1f33d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:53.049659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5399d-d739-40ce-b056-67cbf1f33d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:33:53.050670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-52a5399d-d739-40ce-b056-67cbf1f33d40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:33:53.050670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Aug 07 22:33:53.050670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: Aug 07 22:33:53.051557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5399d-d739-40ce-b056-67cbf1f33d40 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:33:53.064208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 753/1500] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:33:53.068449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682577ef-51f4-4bfc-9e88-db7a3868a8aa admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:33:53.069963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-682577ef-51f4-4bfc-9e88-db7a3868a8aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:53.103485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682577ef-51f4-4bfc-9e88-db7a3868a8aa admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:33:53.195194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77cb17-c4e4-444c-92c9-9986775271fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:33:53.196315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77cb17-c4e4-444c-92c9-9986775271fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:53.196588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77cb17-c4e4-444c-92c9-9986775271fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:53.196860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77cb17-c4e4-444c-92c9-9986775271fc admin admin] PATH_INFO: `/v3/projects/de79fe4dbefc400990db060a8ba8d764/users/691f6fe139994e2dad86feaa8b700152/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:53.218996 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77cb17-c4e4-444c-92c9-9986775271fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=de79fe4dbefc400990db060a8ba8d764, user_id=691f6fe139994e2dad86feaa8b700152, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:33:53.220004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-7a77cb17-c4e4-444c-92c9-9986775271fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:33:53.220004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Aug 07 22:33:53.220004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: Aug 07 22:33:53.221970 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77cb17-c4e4-444c-92c9-9986775271fc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:33:53.310345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 748/1501] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:33:53 2022] PUT /identity/v3/projects/de79fe4dbefc400990db060a8ba8d764/users/691f6fe139994e2dad86feaa8b700152/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:33:53.322969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c08c4-24a8-4b01-a5ef-df73cbbd8535 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:53.327431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c08c4-24a8-4b01-a5ef-df73cbbd8535 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:53.327431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c08c4-24a8-4b01-a5ef-df73cbbd8535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:53.378826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-831c08c4-24a8-4b01-a5ef-df73cbbd8535 None None] MFA Rules not processed for user `691f6fe139994e2dad86feaa8b700152`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:33:53.380339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-831c08c4-24a8-4b01-a5ef-df73cbbd8535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:53.458515 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 754/1502] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:33:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:33:53.472246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53c724-89ce-4a18-b0d4-a3689eb902d4 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:53.472537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53c724-89ce-4a18-b0d4-a3689eb902d4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:53.472820 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53c724-89ce-4a18-b0d4-a3689eb902d4 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:53.474148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 749/1503] 10.222.0.30 () {54 vars in 964 bytes} [Sun Aug 7 22:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:33:53.479265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c4c16-12a0-44b9-ab82-17f8bb845bb8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:53.479546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c4c16-12a0-44b9-ab82-17f8bb845bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:53.479818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c4c16-12a0-44b9-ab82-17f8bb845bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:53.512542 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:33:53.556649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9f9c4c16-12a0-44b9-ab82-17f8bb845bb8 None None] MFA Rules not processed for user `407b8483d4f34be19d2c7c7b6013b313`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:33:53.557879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9f9c4c16-12a0-44b9-ab82-17f8bb845bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:53.633740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 755/1504] 10.222.0.30 () {58 vars in 1061 bytes} [Sun Aug 7 22:33:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:33:53.641419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53c724-89ce-4a18-b0d4-a3689eb902d4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:33:53.642955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5a53c724-89ce-4a18-b0d4-a3689eb902d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:53.691864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:33:53.745658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53c724-89ce-4a18-b0d4-a3689eb902d4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:33:53.819013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083c059-562c-4acb-9f85-74f25b7a4724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:33:53.819622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083c059-562c-4acb-9f85-74f25b7a4724 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:53.819902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083c059-562c-4acb-9f85-74f25b7a4724 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:53.820208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083c059-562c-4acb-9f85-74f25b7a4724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:53.822610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a083c059-562c-4acb-9f85-74f25b7a4724 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:53.923245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083c059-562c-4acb-9f85-74f25b7a4724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:33:53.924699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083c059-562c-4acb-9f85-74f25b7a4724 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:33:53.928237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:33:53.943699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 750/1505] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:33:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:33:53.988766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73e366-acf4-4ec1-bd4e-a35afdef50ea None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:53.989245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73e366-acf4-4ec1-bd4e-a35afdef50ea None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:53.989449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73e366-acf4-4ec1-bd4e-a35afdef50ea None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:53.990702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 756/1506] 10.222.0.30 () {52 vars in 830 bytes} [Sun Aug 7 22:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:33:53.995219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083c059-562c-4acb-9f85-74f25b7a4724 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:33:53.996478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a083c059-562c-4acb-9f85-74f25b7a4724 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:54.097153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083c059-562c-4acb-9f85-74f25b7a4724 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:33:54.189607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:33:54.190262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:54.190545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:54.190814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] PATH_INFO: `/v3/users/691f6fe139994e2dad86feaa8b700152` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:54.203375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=691f6fe139994e2dad86feaa8b700152)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:33:54.204434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:33:54.204434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Aug 07 22:33:54.204434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: Aug 07 22:33:54.205641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:33:54.215927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:33:54.411094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691f6fe139994e2dad86feaa8b700152'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:33:54.481491 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691f6fe139994e2dad86feaa8b700152'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:33:54.504760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691f6fe139994e2dad86feaa8b700152'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:33:54.505316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691f6fe139994e2dad86feaa8b700152'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:33:54.505741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691f6fe139994e2dad86feaa8b700152'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:33:54.525587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 751/1507] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:33:53 2022] DELETE /identity/v3/users/691f6fe139994e2dad86feaa8b700152 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:33:54.530174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276fc5e-c60e-4d6c-b21f-3bbfd40aec37 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:54.531879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276fc5e-c60e-4d6c-b21f-3bbfd40aec37 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:54.531879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276fc5e-c60e-4d6c-b21f-3bbfd40aec37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:54.588610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b276fc5e-c60e-4d6c-b21f-3bbfd40aec37 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:33:54.589826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b276fc5e-c60e-4d6c-b21f-3bbfd40aec37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:54.666077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:33:54.678718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 757/1508] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:33:54.688906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:33:54.690172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:54.791781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25357b3-a40a-4881-a429-9a629d7acd9d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:33:54.845656 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:33:54.864074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4238dd-5935-481d-af66-93a69a29999f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:33:54.864699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4238dd-5935-481d-af66-93a69a29999f service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:54.864969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4238dd-5935-481d-af66-93a69a29999f service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:54.865246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4238dd-5935-481d-af66-93a69a29999f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:54.866888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3e4238dd-5935-481d-af66-93a69a29999f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:54.978254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4238dd-5935-481d-af66-93a69a29999f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:33:54.979633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4238dd-5935-481d-af66-93a69a29999f service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:33:54.998154 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 752/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:33:55.437209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276fc5e-c60e-4d6c-b21f-3bbfd40aec37 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:33:55.438461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b276fc5e-c60e-4d6c-b21f-3bbfd40aec37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:33:55.451104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276fc5e-c60e-4d6c-b21f-3bbfd40aec37 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:33:55.548168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:33:55.548796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:33:55.549079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:33:55.549347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] PATH_INFO: `/v3/projects/de79fe4dbefc400990db060a8ba8d764` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:33:55.559874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de79fe4dbefc400990db060a8ba8d764)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:33:55.560924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:33:55.560924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Aug 07 22:33:55.560924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: Aug 07 22:33:55.562116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:33:55.663561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de79fe4dbefc400990db060a8ba8d764', 'request_id': 'req-7befbd8d-312e-4c67-8093-446a09106296'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:33:55.677038 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de79fe4dbefc400990db060a8ba8d764', 'request_id': 'req-7befbd8d-312e-4c67-8093-446a09106296'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:33:55.738452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de79fe4dbefc400990db060a8ba8d764', 'request_id': 'req-7befbd8d-312e-4c67-8093-446a09106296'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:33:55.762183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] The token cache is being invalidate because project de79fe4dbefc400990db060a8ba8d764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:33:55.762469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:33:55.762897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-7befbd8d-312e-4c67-8093-446a09106296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:33:55.764321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 758/1510] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:33:55 2022] DELETE /identity/v3/projects/de79fe4dbefc400990db060a8ba8d764 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:34:04.995849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=62 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:34:05.759938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:35:23.385495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992725ff-1680-4ac6-90a6-90108a2637e8 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:23.386081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992725ff-1680-4ac6-90a6-90108a2637e8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:23.386081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992725ff-1680-4ac6-90a6-90108a2637e8 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:23.387416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:35:23.392373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db5dc9-00b5-4ae2-9a76-7b4c2983d7ee None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:23.392813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db5dc9-00b5-4ae2-9a76-7b4c2983d7ee None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:23.392938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db5dc9-00b5-4ae2-9a76-7b4c2983d7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:23.472594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-54db5dc9-00b5-4ae2-9a76-7b4c2983d7ee None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:35:23.473803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-54db5dc9-00b5-4ae2-9a76-7b4c2983d7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:23.576402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:35:23.735680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992725ff-1680-4ac6-90a6-90108a2637e8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:35:23.739346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-992725ff-1680-4ac6-90a6-90108a2637e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:23.858113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992725ff-1680-4ac6-90a6-90108a2637e8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:35:23.863791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40b272-a4e6-42c3-9f89-fc0318bba037 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:35:23.865300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b272-a4e6-42c3-9f89-fc0318bba037 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:23.865300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b272-a4e6-42c3-9f89-fc0318bba037 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:23.865300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b272-a4e6-42c3-9f89-fc0318bba037 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:23.873696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40b272-a4e6-42c3-9f89-fc0318bba037 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:35:23.875287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40b272-a4e6-42c3-9f89-fc0318bba037 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:35:23.885845 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 754/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:35:24.726484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2c92c-1e08-4f38-8a89-3c51085076fa None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:24.727027 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2c92c-1e08-4f38-8a89-3c51085076fa None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:24.727027 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2c92c-1e08-4f38-8a89-3c51085076fa None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:24.728576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:35:24.733419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e660b-209b-42b5-8dcc-136877351964 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:24.733882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e660b-209b-42b5-8dcc-136877351964 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:24.733990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e660b-209b-42b5-8dcc-136877351964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:24.753154 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e44e660b-209b-42b5-8dcc-136877351964 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:35:24.754357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e44e660b-209b-42b5-8dcc-136877351964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:24.763499 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:35:24.912926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2c92c-1e08-4f38-8a89-3c51085076fa None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:35:24.928328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2c92c-1e08-4f38-8a89-3c51085076fa None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:35:24.934487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b3630-eb47-45b7-9daa-0a8347e98766 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:35:24.935677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b3630-eb47-45b7-9daa-0a8347e98766 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:24.935677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b3630-eb47-45b7-9daa-0a8347e98766 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:24.935677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b3630-eb47-45b7-9daa-0a8347e98766 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:24.944446 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b3630-eb47-45b7-9daa-0a8347e98766 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:35:24.946056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b3630-eb47-45b7-9daa-0a8347e98766 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:35:24.955861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 761/1516] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:35:25.446513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822ee6b-e346-4c73-b40d-c993fde6442f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:25.447345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822ee6b-e346-4c73-b40d-c993fde6442f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:25.447672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822ee6b-e346-4c73-b40d-c993fde6442f None None] PATH_INFO: `/v3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:25.449019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 756/1517] 10.222.0.27 () {54 vars in 925 bytes} [Sun Aug 7 22:35:25 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:35:25.458765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bb68f-7f75-43d8-a534-8a66b0567383 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:25.459049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bb68f-7f75-43d8-a534-8a66b0567383 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:25.459322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bb68f-7f75-43d8-a534-8a66b0567383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:25.517819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-471bb68f-7f75-43d8-a534-8a66b0567383 None None] MFA Rules not processed for user `db1d7e39e8c14f5db40efa743fc3b652`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:35:25.519028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-471bb68f-7f75-43d8-a534-8a66b0567383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:25.536200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:35:25.590602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 762/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Sun Aug 7 22:35:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:35:29.674166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec6e16-8414-48d0-8bb9-e0f10a5d7678 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:29.674712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec6e16-8414-48d0-8bb9-e0f10a5d7678 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:29.674712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec6e16-8414-48d0-8bb9-e0f10a5d7678 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:29.676142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 757/1519] 10.222.0.30 () {54 vars in 961 bytes} [Sun Aug 7 22:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:35:29.681729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863aea5-8243-4f61-8e33-70fc80fe6c8b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:29.682160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863aea5-8243-4f61-8e33-70fc80fe6c8b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:29.682268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863aea5-8243-4f61-8e33-70fc80fe6c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:29.702485 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9863aea5-8243-4f61-8e33-70fc80fe6c8b None None] MFA Rules not processed for user `db1d7e39e8c14f5db40efa743fc3b652`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:35:29.703680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9863aea5-8243-4f61-8e33-70fc80fe6c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:29.712032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 763/1520] 10.222.0.30 () {58 vars in 1058 bytes} [Sun Aug 7 22:35:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:35:29.719927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec6e16-8414-48d0-8bb9-e0f10a5d7678 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:35:29.733885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec6e16-8414-48d0-8bb9-e0f10a5d7678 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:35:29.738737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252f18f-2ca5-4850-b029-16034ba698ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:35:29.739320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252f18f-2ca5-4850-b029-16034ba698ca service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:29.739598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252f18f-2ca5-4850-b029-16034ba698ca service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:29.739868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252f18f-2ca5-4850-b029-16034ba698ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:29.748256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252f18f-2ca5-4850-b029-16034ba698ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:35:29.749844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252f18f-2ca5-4850-b029-16034ba698ca service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:35:29.759460 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 758/1521] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:35:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:35:36.186263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1c0af-ed91-442b-a7fd-5c7821f3e84e None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:36.186842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1c0af-ed91-442b-a7fd-5c7821f3e84e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:36.186842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1c0af-ed91-442b-a7fd-5c7821f3e84e None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:36.188196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:35:36.192935 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec93384-fac4-4960-aed6-37e7b18ea16c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:36.193321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec93384-fac4-4960-aed6-37e7b18ea16c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:36.193448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec93384-fac4-4960-aed6-37e7b18ea16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:36.213783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-5ec93384-fac4-4960-aed6-37e7b18ea16c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:35:36.214995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5ec93384-fac4-4960-aed6-37e7b18ea16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:36.222835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:35:36.374907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1c0af-ed91-442b-a7fd-5c7821f3e84e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:35:36.390600 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1c0af-ed91-442b-a7fd-5c7821f3e84e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:35:36.395960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aab9ee-b2b5-4825-9e24-3bbed6c88724 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:35:36.396575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aab9ee-b2b5-4825-9e24-3bbed6c88724 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:36.396845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aab9ee-b2b5-4825-9e24-3bbed6c88724 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:36.397113 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aab9ee-b2b5-4825-9e24-3bbed6c88724 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:36.405282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aab9ee-b2b5-4825-9e24-3bbed6c88724 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:35:36.406919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aab9ee-b2b5-4825-9e24-3bbed6c88724 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:35:36.416312 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 765/1524] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:35:37.230700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a899e9f-e6f2-4146-adac-e80a39d1ca52 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:37.231214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a899e9f-e6f2-4146-adac-e80a39d1ca52 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:37.231214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a899e9f-e6f2-4146-adac-e80a39d1ca52 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:37.232580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 22:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:35:37.237223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd8267-bf87-4816-8590-85d1570f35ad None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:37.237624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd8267-bf87-4816-8590-85d1570f35ad None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:37.237765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd8267-bf87-4816-8590-85d1570f35ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:37.256954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-11cd8267-bf87-4816-8590-85d1570f35ad None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:35:37.258144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-11cd8267-bf87-4816-8590-85d1570f35ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:37.266057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 22:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:35:37.420220 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a899e9f-e6f2-4146-adac-e80a39d1ca52 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:35:37.435178 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a899e9f-e6f2-4146-adac-e80a39d1ca52 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:35:37.440626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8743a-7590-4801-b344-9c84b1e0957a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:35:37.441218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8743a-7590-4801-b344-9c84b1e0957a service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:37.441512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8743a-7590-4801-b344-9c84b1e0957a service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:37.441771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8743a-7590-4801-b344-9c84b1e0957a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:37.449984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8743a-7590-4801-b344-9c84b1e0957a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:35:37.451555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8743a-7590-4801-b344-9c84b1e0957a service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:35:37.460741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 761/1527] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:35:41.214378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf3933-0a79-4596-986a-8b51f63af43e None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:41.214882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf3933-0a79-4596-986a-8b51f63af43e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:41.214882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf3933-0a79-4596-986a-8b51f63af43e None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:41.216229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:35:41.221480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cb1d4-d18f-4e48-a04a-ce7e6975247b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:41.221881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cb1d4-d18f-4e48-a04a-ce7e6975247b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:41.222028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cb1d4-d18f-4e48-a04a-ce7e6975247b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:41.242570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-292cb1d4-d18f-4e48-a04a-ce7e6975247b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:35:41.244501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-292cb1d4-d18f-4e48-a04a-ce7e6975247b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:41.252338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:35:41.324672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf3933-0a79-4596-986a-8b51f63af43e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:35:41.326379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-52bf3933-0a79-4596-986a-8b51f63af43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:41.404589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf3933-0a79-4596-986a-8b51f63af43e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:35:41.408202 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57307506-a259-4ad2-abe3-a41b6a56ccd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:35:41.408801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57307506-a259-4ad2-abe3-a41b6a56ccd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:41.409077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57307506-a259-4ad2-abe3-a41b6a56ccd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:41.409348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57307506-a259-4ad2-abe3-a41b6a56ccd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:41.418756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57307506-a259-4ad2-abe3-a41b6a56ccd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:35:41.420222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57307506-a259-4ad2-abe3-a41b6a56ccd1 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:35:41.429371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 768/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:35:43.433013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117362f9-faf9-456c-8d0b-e99e5945c61d None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:43.433585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117362f9-faf9-456c-8d0b-e99e5945c61d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:43.433585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117362f9-faf9-456c-8d0b-e99e5945c61d None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:43.434936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:35:43.439978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889035-05a6-4551-8016-7ae58833a3e1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:43.440376 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889035-05a6-4551-8016-7ae58833a3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:43.440567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889035-05a6-4551-8016-7ae58833a3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:43.459773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0e889035-05a6-4551-8016-7ae58833a3e1 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:35:43.461016 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0e889035-05a6-4551-8016-7ae58833a3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:43.468988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:35:43.505741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117362f9-faf9-456c-8d0b-e99e5945c61d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:35:43.517055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117362f9-faf9-456c-8d0b-e99e5945c61d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:35:43.520597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e041b-dd8c-4637-b4fe-a5058a577119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:35:43.521185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e041b-dd8c-4637-b4fe-a5058a577119 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:43.521464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e041b-dd8c-4637-b4fe-a5058a577119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:43.521737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e041b-dd8c-4637-b4fe-a5058a577119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:43.530826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e041b-dd8c-4637-b4fe-a5058a577119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:35:43.532285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e041b-dd8c-4637-b4fe-a5058a577119 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:35:43.542209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 764/1533] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:35:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:35:47.128044 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1838a0-a96c-4a37-ab03-a53bf928a0f5 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:47.128634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1838a0-a96c-4a37-ab03-a53bf928a0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:47.128634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1838a0-a96c-4a37-ab03-a53bf928a0f5 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:47.129999 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:35:47.135880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de6e8-cd5a-4a45-9f9a-0212b043024e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:47.136336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de6e8-cd5a-4a45-9f9a-0212b043024e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:47.136520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de6e8-cd5a-4a45-9f9a-0212b043024e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:47.143958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=81 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:35:47.156162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8e6de6e8-cd5a-4a45-9f9a-0212b043024e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:35:47.157940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8e6de6e8-cd5a-4a45-9f9a-0212b043024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:47.165741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:35:47.468444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1838a0-a96c-4a37-ab03-a53bf928a0f5 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:35:47.470170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6c1838a0-a96c-4a37-ab03-a53bf928a0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:47.534250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:35:47.547454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1838a0-a96c-4a37-ab03-a53bf928a0f5 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:35:47.550928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab9e84-1fad-48ef-8ff0-e40c23d56214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:35:47.551522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab9e84-1fad-48ef-8ff0-e40c23d56214 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:47.551796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab9e84-1fad-48ef-8ff0-e40c23d56214 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:47.552099 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab9e84-1fad-48ef-8ff0-e40c23d56214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:47.560755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab9e84-1fad-48ef-8ff0-e40c23d56214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:35:47.562208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab9e84-1fad-48ef-8ff0-e40c23d56214 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:35:47.571305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 771/1536] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:35:49.575105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b50b61-f74f-4cef-8904-8447dedad6e8 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:49.575772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b50b61-f74f-4cef-8904-8447dedad6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:49.575772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b50b61-f74f-4cef-8904-8447dedad6e8 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:49.577004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 22:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:35:49.582064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42774d-d456-4bcf-9b69-4f4c2ce005b5 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:49.582525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42774d-d456-4bcf-9b69-4f4c2ce005b5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:49.582641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42774d-d456-4bcf-9b69-4f4c2ce005b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:49.602451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9a42774d-d456-4bcf-9b69-4f4c2ce005b5 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:35:49.603751 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9a42774d-d456-4bcf-9b69-4f4c2ce005b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:49.611701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 22:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:35:49.913933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b50b61-f74f-4cef-8904-8447dedad6e8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:35:49.916392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b1b50b61-f74f-4cef-8904-8447dedad6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:35:49.929112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b50b61-f74f-4cef-8904-8447dedad6e8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:35:49.932729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5be49-22ac-476f-9268-c024cb4c6edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:35:49.933317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5be49-22ac-476f-9268-c024cb4c6edf service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:35:49.933591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5be49-22ac-476f-9268-c024cb4c6edf service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:35:49.933861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5be49-22ac-476f-9268-c024cb4c6edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:35:49.942877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5be49-22ac-476f-9268-c024cb4c6edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:35:49.944310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5be49-22ac-476f-9268-c024cb4c6edf service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:35:49.953802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 767/1539] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:35:59.597740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=11 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:35:59.949619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=8 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:14.097839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e69cdb-dc40-4cf2-9845-fb5459b71799 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.098494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e69cdb-dc40-4cf2-9845-fb5459b71799 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.098494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e69cdb-dc40-4cf2-9845-fb5459b71799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.126587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d2e69cdb-dc40-4cf2-9845-fb5459b71799 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.127867 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d2e69cdb-dc40-4cf2-9845-fb5459b71799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.142169 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 773/1540] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.150535 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9877e-d537-4e8f-ba63-5b7627569f0d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.151155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9877e-d537-4e8f-ba63-5b7627569f0d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.151155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9877e-d537-4e8f-ba63-5b7627569f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.181052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-57f9877e-d537-4e8f-ba63-5b7627569f0d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.182289 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-57f9877e-d537-4e8f-ba63-5b7627569f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.195122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 768/1541] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.200674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e69cdb-dc40-4cf2-9845-fb5459b71799 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:14.216148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e69cdb-dc40-4cf2-9845-fb5459b71799 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:14.225968 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1c5e7-0a0d-485b-a283-7b3ab566a5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:14.226605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1c5e7-0a0d-485b-a283-7b3ab566a5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.226886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1c5e7-0a0d-485b-a283-7b3ab566a5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.227160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1c5e7-0a0d-485b-a283-7b3ab566a5f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.228666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1c5e7-0a0d-485b-a283-7b3ab566a5f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:14.236665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1c5e7-0a0d-485b-a283-7b3ab566a5f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:14.236665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1c5e7-0a0d-485b-a283-7b3ab566a5f3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:14.253595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 774/1542] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:14.263210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9877e-d537-4e8f-ba63-5b7627569f0d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:14.273161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9877e-d537-4e8f-ba63-5b7627569f0d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:14.282477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aa0d0-ea1b-4c45-996a-dbc84d42912b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:14.283100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa0d0-ea1b-4c45-996a-dbc84d42912b admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.283375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa0d0-ea1b-4c45-996a-dbc84d42912b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.283647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa0d0-ea1b-4c45-996a-dbc84d42912b admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.284308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6aa0d0-ea1b-4c45-996a-dbc84d42912b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:14.286274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6aa0d0-ea1b-4c45-996a-dbc84d42912b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:14.301307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c4910-34f2-4428-a3bd-c9090da18c9e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.307901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c4910-34f2-4428-a3bd-c9090da18c9e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.307901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c4910-34f2-4428-a3bd-c9090da18c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.337384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a45c4910-34f2-4428-a3bd-c9090da18c9e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.338765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a45c4910-34f2-4428-a3bd-c9090da18c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.356107 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 775/1543] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.366282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183213c-1063-4d18-8fae-ca8ddd43afef None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.366622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183213c-1063-4d18-8fae-ca8ddd43afef None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.366848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183213c-1063-4d18-8fae-ca8ddd43afef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.398076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 769/1544] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:14.398412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f183213c-1063-4d18-8fae-ca8ddd43afef None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.403803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aa0d0-ea1b-4c45-996a-dbc84d42912b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:14.404176 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f183213c-1063-4d18-8fae-ca8ddd43afef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.540220 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aa0d0-ea1b-4c45-996a-dbc84d42912b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:14.562307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae6e3b-3e56-4409-add8-ed4bb8860594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:14.562588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 776/1545] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.563495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae6e3b-3e56-4409-add8-ed4bb8860594 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.563711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabaf96-b433-4c70-b805-42f0a86ad230 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.563986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabaf96-b433-4c70-b805-42f0a86ad230 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.564291 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabaf96-b433-4c70-b805-42f0a86ad230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.564836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae6e3b-3e56-4409-add8-ed4bb8860594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.565342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae6e3b-3e56-4409-add8-ed4bb8860594 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.566190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae6e3b-3e56-4409-add8-ed4bb8860594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:14.567463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-94ae6e3b-3e56-4409-add8-ed4bb8860594 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:37:14.567463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Aug 07 22:37:14.567463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: Aug 07 22:37:14.569245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae6e3b-3e56-4409-add8-ed4bb8860594 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:14.595253 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ebabaf96-b433-4c70-b805-42f0a86ad230 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.596800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ebabaf96-b433-4c70-b805-42f0a86ad230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.606427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 777/1546] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.613808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8af3a8-4b9b-4fad-99ab-24cafdcbb32e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.613808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8af3a8-4b9b-4fad-99ab-24cafdcbb32e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.613808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8af3a8-4b9b-4fad-99ab-24cafdcbb32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.627658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-af8af3a8-4b9b-4fad-99ab-24cafdcbb32e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.628900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-af8af3a8-4b9b-4fad-99ab-24cafdcbb32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.638055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 778/1547] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.639325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53680a44-54fb-44bb-8cc2-4fc554f848b9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.642217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53680a44-54fb-44bb-8cc2-4fc554f848b9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.642217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53680a44-54fb-44bb-8cc2-4fc554f848b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.659266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-53680a44-54fb-44bb-8cc2-4fc554f848b9 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.660598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-53680a44-54fb-44bb-8cc2-4fc554f848b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.668317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 779/1548] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.672673 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53680a44-54fb-44bb-8cc2-4fc554f848b9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:14.678557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53680a44-54fb-44bb-8cc2-4fc554f848b9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:14.683784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a719603-6ef4-4484-8bb5-028ee6df8b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:14.684396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a719603-6ef4-4484-8bb5-028ee6df8b40 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.684671 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a719603-6ef4-4484-8bb5-028ee6df8b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.684943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a719603-6ef4-4484-8bb5-028ee6df8b40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.685577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a719603-6ef4-4484-8bb5-028ee6df8b40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:14.685880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a719603-6ef4-4484-8bb5-028ee6df8b40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:14.687466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a719603-6ef4-4484-8bb5-028ee6df8b40 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:14.697919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 770/1549] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/users => generated 283 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:37:14.698084 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed77ac4-ce00-4f8e-9089-af0b45e26d5d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.703625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed77ac4-ce00-4f8e-9089-af0b45e26d5d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.703625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed77ac4-ce00-4f8e-9089-af0b45e26d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.703842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 780/1550] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:14.703842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242668-7cd9-48cd-b5ce-d6e2dc0d4bc7 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.703842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242668-7cd9-48cd-b5ce-d6e2dc0d4bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.703842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242668-7cd9-48cd-b5ce-d6e2dc0d4bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.723411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2ed77ac4-ce00-4f8e-9089-af0b45e26d5d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.724719 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2ed77ac4-ce00-4f8e-9089-af0b45e26d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.724922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-dc242668-7cd9-48cd-b5ce-d6e2dc0d4bc7 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.726140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dc242668-7cd9-48cd-b5ce-d6e2dc0d4bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.732999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 771/1551] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.734301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3423da-b13d-40a1-930c-d73de955cec7 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.741561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3423da-b13d-40a1-930c-d73de955cec7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.741738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 781/1552] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.741851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3423da-b13d-40a1-930c-d73de955cec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.741950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2cf92-9ea6-4a0a-b72e-bf3e710aaff7 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.741950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2cf92-9ea6-4a0a-b72e-bf3e710aaff7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.741950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2cf92-9ea6-4a0a-b72e-bf3e710aaff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.756436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-02f2cf92-9ea6-4a0a-b72e-bf3e710aaff7 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.757317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7a3423da-b13d-40a1-930c-d73de955cec7 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.757884 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-02f2cf92-9ea6-4a0a-b72e-bf3e710aaff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.758482 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7a3423da-b13d-40a1-930c-d73de955cec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.766699 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 782/1553] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.772217 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 772/1554] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.772314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76709b50-11b8-4dc6-8551-ebc19afe4188 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.772314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76709b50-11b8-4dc6-8551-ebc19afe4188 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.773254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28702b-a91e-4191-aae0-f9bbe76dd7b9 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.773367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76709b50-11b8-4dc6-8551-ebc19afe4188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.773456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28702b-a91e-4191-aae0-f9bbe76dd7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.773456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28702b-a91e-4191-aae0-f9bbe76dd7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.789107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-dc28702b-a91e-4191-aae0-f9bbe76dd7b9 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.790932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-dc28702b-a91e-4191-aae0-f9bbe76dd7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.792634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-76709b50-11b8-4dc6-8551-ebc19afe4188 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.793806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-76709b50-11b8-4dc6-8551-ebc19afe4188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.799120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 773/1555] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.799894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28702b-a91e-4191-aae0-f9bbe76dd7b9 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:14.802024 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 783/1556] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.805116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76709b50-11b8-4dc6-8551-ebc19afe4188 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:14.812687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28702b-a91e-4191-aae0-f9bbe76dd7b9 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:14.815512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19466985-e3c4-4173-bdf6-c38f3f293dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:14.815928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76709b50-11b8-4dc6-8551-ebc19afe4188 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:14.816146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19466985-e3c4-4173-bdf6-c38f3f293dea admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.816415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19466985-e3c4-4173-bdf6-c38f3f293dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.816680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19466985-e3c4-4173-bdf6-c38f3f293dea admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.817315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19466985-e3c4-4173-bdf6-c38f3f293dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:14.818891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19466985-e3c4-4173-bdf6-c38f3f293dea admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:14.820725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23470be1-8096-4e72-ab4a-6811aa7d0e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:14.821336 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23470be1-8096-4e72-ab4a-6811aa7d0e61 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.821945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23470be1-8096-4e72-ab4a-6811aa7d0e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.822313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23470be1-8096-4e72-ab4a-6811aa7d0e61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.822942 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23470be1-8096-4e72-ab4a-6811aa7d0e61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:14.824001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-23470be1-8096-4e72-ab4a-6811aa7d0e61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:37:14.824001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Aug 07 22:37:14.824001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: Aug 07 22:37:14.825293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23470be1-8096-4e72-ab4a-6811aa7d0e61 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:14.829179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 774/1557] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:14.830343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc47e40-e981-4a49-8e59-35ece6460648 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.830632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc47e40-e981-4a49-8e59-35ece6460648 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.830908 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc47e40-e981-4a49-8e59-35ece6460648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.849601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4cc47e40-e981-4a49-8e59-35ece6460648 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.853113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4cc47e40-e981-4a49-8e59-35ece6460648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.859103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 775/1558] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.860456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659bf40-2e58-4909-aea8-a2ec2261266c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.860739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659bf40-2e58-4909-aea8-a2ec2261266c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.861020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659bf40-2e58-4909-aea8-a2ec2261266c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.880036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4659bf40-2e58-4909-aea8-a2ec2261266c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.881215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4659bf40-2e58-4909-aea8-a2ec2261266c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.889591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 776/1559] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:14.890416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659bf40-2e58-4909-aea8-a2ec2261266c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:14.900214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659bf40-2e58-4909-aea8-a2ec2261266c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:14.904865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af55c8-6180-4801-9aa0-a3940f3b2e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:14.905489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af55c8-6180-4801-9aa0-a3940f3b2e76 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.905772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af55c8-6180-4801-9aa0-a3940f3b2e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.906046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af55c8-6180-4801-9aa0-a3940f3b2e76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.906681 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af55c8-6180-4801-9aa0-a3940f3b2e76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:14.906985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af55c8-6180-4801-9aa0-a3940f3b2e76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:14.908003 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-34af55c8-6180-4801-9aa0-a3940f3b2e76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:37:14.908003 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Aug 07 22:37:14.908003 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: Aug 07 22:37:14.908911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af55c8-6180-4801-9aa0-a3940f3b2e76 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:14.916210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 784/1560] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:14.919511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3143e-56fe-4e29-81b9-b4707d4a67cc None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:14.919511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3143e-56fe-4e29-81b9-b4707d4a67cc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:14.919511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3143e-56fe-4e29-81b9-b4707d4a67cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:14.921115 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 777/1561] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:14.928652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af55c8-6180-4801-9aa0-a3940f3b2e76 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:14.938017 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-57e3143e-56fe-4e29-81b9-b4707d4a67cc None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:14.939245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-57e3143e-56fe-4e29-81b9-b4707d4a67cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:14.943738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af55c8-6180-4801-9aa0-a3940f3b2e76 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:14.991243 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:14.999098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:15.001256 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 785/1562] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:15.002231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3143e-56fe-4e29-81b9-b4707d4a67cc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.005980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5852d9-3d13-4330-96d5-95ca3d2b87b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.006598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5852d9-3d13-4330-96d5-95ca3d2b87b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.007128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5852d9-3d13-4330-96d5-95ca3d2b87b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.007591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5852d9-3d13-4330-96d5-95ca3d2b87b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.008274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5852d9-3d13-4330-96d5-95ca3d2b87b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:15.009233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5852d9-3d13-4330-96d5-95ca3d2b87b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.010800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5852d9-3d13-4330-96d5-95ca3d2b87b9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.012223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3143e-56fe-4e29-81b9-b4707d4a67cc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.021173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7d014-1e77-45e3-b97d-055b7a4370b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.021173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d014-1e77-45e3-b97d-055b7a4370b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.021173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d014-1e77-45e3-b97d-055b7a4370b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.021173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d014-1e77-45e3-b97d-055b7a4370b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.021635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7d014-1e77-45e3-b97d-055b7a4370b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:15.021635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7d014-1e77-45e3-b97d-055b7a4370b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.021635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7d014-1e77-45e3-b97d-055b7a4370b8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.122214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 786/1563] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:15.122868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 778/1564] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:15.123362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7d014-1e77-45e3-b97d-055b7a4370b8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.123479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5852d9-3d13-4330-96d5-95ca3d2b87b9 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.134037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5852d9-3d13-4330-96d5-95ca3d2b87b9 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.139287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63cca3-0a43-4559-81b7-c2fb6b8a6bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.139865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7d014-1e77-45e3-b97d-055b7a4370b8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.139995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63cca3-0a43-4559-81b7-c2fb6b8a6bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.140221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63cca3-0a43-4559-81b7-c2fb6b8a6bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.140494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63cca3-0a43-4559-81b7-c2fb6b8a6bd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.141142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63cca3-0a43-4559-81b7-c2fb6b8a6bd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:15.141447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63cca3-0a43-4559-81b7-c2fb6b8a6bd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.143039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63cca3-0a43-4559-81b7-c2fb6b8a6bd1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.144760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7caded-c12d-4eb7-b0d3-3a21252c8fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.147491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7caded-c12d-4eb7-b0d3-3a21252c8fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.147491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7caded-c12d-4eb7-b0d3-3a21252c8fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.147491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7caded-c12d-4eb7-b0d3-3a21252c8fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.147491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7caded-c12d-4eb7-b0d3-3a21252c8fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.148379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7caded-c12d-4eb7-b0d3-3a21252c8fdf admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.156933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 779/1565] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:15.157893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63cca3-0a43-4559-81b7-c2fb6b8a6bd1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.164868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 787/1566] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:15.166824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7caded-c12d-4eb7-b0d3-3a21252c8fdf admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.168643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63cca3-0a43-4559-81b7-c2fb6b8a6bd1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.173219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef265c3-f375-4a5b-bb78-6301cb323ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.173811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef265c3-f375-4a5b-bb78-6301cb323ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.174088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef265c3-f375-4a5b-bb78-6301cb323ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.174417 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef265c3-f375-4a5b-bb78-6301cb323ec9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.174988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef265c3-f375-4a5b-bb78-6301cb323ec9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:15.175292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef265c3-f375-4a5b-bb78-6301cb323ec9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.176899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef265c3-f375-4a5b-bb78-6301cb323ec9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.177144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7caded-c12d-4eb7-b0d3-3a21252c8fdf admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.182538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75159f-754c-42aa-ab75-c8c24c956078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.183124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75159f-754c-42aa-ab75-c8c24c956078 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.183405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75159f-754c-42aa-ab75-c8c24c956078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.183676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75159f-754c-42aa-ab75-c8c24c956078 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.184354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75159f-754c-42aa-ab75-c8c24c956078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.186244 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75159f-754c-42aa-ab75-c8c24c956078 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.188381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 780/1567] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:15.189227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef265c3-f375-4a5b-bb78-6301cb323ec9 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.200967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef265c3-f375-4a5b-bb78-6301cb323ec9 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.205966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dad79e-2e87-43d4-aa6f-b49a07789563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.206570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dad79e-2e87-43d4-aa6f-b49a07789563 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.206970 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dad79e-2e87-43d4-aa6f-b49a07789563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.207245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dad79e-2e87-43d4-aa6f-b49a07789563 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.207863 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dad79e-2e87-43d4-aa6f-b49a07789563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.210100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dad79e-2e87-43d4-aa6f-b49a07789563 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.301179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 781/1568] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:15 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:15.301179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dad79e-2e87-43d4-aa6f-b49a07789563 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.348231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 788/1569] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:15 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:15.349051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75159f-754c-42aa-ab75-c8c24c956078 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.358801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75159f-754c-42aa-ab75-c8c24c956078 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.363689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffb065-08b1-4c61-85bc-79120ef28524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.364141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dad79e-2e87-43d4-aa6f-b49a07789563 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.364339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffb065-08b1-4c61-85bc-79120ef28524 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.364619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffb065-08b1-4c61-85bc-79120ef28524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.364892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffb065-08b1-4c61-85bc-79120ef28524 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.365512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bffb065-08b1-4c61-85bc-79120ef28524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.367420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bffb065-08b1-4c61-85bc-79120ef28524 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.369259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f511f5-00f4-435e-908f-152abf22c78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.369857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f511f5-00f4-435e-908f-152abf22c78e admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.370133 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f511f5-00f4-435e-908f-152abf22c78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.370408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f511f5-00f4-435e-908f-152abf22c78e admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.371052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f511f5-00f4-435e-908f-152abf22c78e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:15.371355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f511f5-00f4-435e-908f-152abf22c78e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.373021 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f511f5-00f4-435e-908f-152abf22c78e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.385442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 782/1570] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:15.386168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f511f5-00f4-435e-908f-152abf22c78e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.395885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f511f5-00f4-435e-908f-152abf22c78e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.400665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db5709-4849-4222-b37b-846a9b832f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.401301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db5709-4849-4222-b37b-846a9b832f73 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.401621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db5709-4849-4222-b37b-846a9b832f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.401892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db5709-4849-4222-b37b-846a9b832f73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.402523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db5709-4849-4222-b37b-846a9b832f73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.404421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db5709-4849-4222-b37b-846a9b832f73 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.435127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 789/1571] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:15 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:15.436171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffb065-08b1-4c61-85bc-79120ef28524 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.498038 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 783/1572] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:15 2022] POST /identity/v3/projects => generated 391 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:15.499381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db5709-4849-4222-b37b-846a9b832f73 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.505480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffb065-08b1-4c61-85bc-79120ef28524 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.562961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26decf3-7887-4e4b-8721-6948d4a84b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.563577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26decf3-7887-4e4b-8721-6948d4a84b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.564249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db5709-4849-4222-b37b-846a9b832f73 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.564519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26decf3-7887-4e4b-8721-6948d4a84b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.564799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26decf3-7887-4e4b-8721-6948d4a84b5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.565426 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26decf3-7887-4e4b-8721-6948d4a84b5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.567333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26decf3-7887-4e4b-8721-6948d4a84b5e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.568897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772a29a-0ebb-4e6c-bf78-32f0ce926844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.569583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772a29a-0ebb-4e6c-bf78-32f0ce926844 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.569864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772a29a-0ebb-4e6c-bf78-32f0ce926844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.570210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772a29a-0ebb-4e6c-bf78-32f0ce926844 admin admin] PATH_INFO: `/v3/projects/b89549eee7fd4f2ba1657576213f254a/users/d89c4aa62b3f4247a2d76011f7b2d12c/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.585249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772a29a-0ebb-4e6c-bf78-32f0ce926844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b89549eee7fd4f2ba1657576213f254a, user_id=d89c4aa62b3f4247a2d76011f7b2d12c, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.588041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772a29a-0ebb-4e6c-bf78-32f0ce926844 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.591215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:15.651460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 790/1573] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:15 2022] POST /identity/v3/projects => generated 403 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:15.652239 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26decf3-7887-4e4b-8721-6948d4a84b5e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.680142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 784/1574] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:15 2022] PUT /identity/v3/projects/b89549eee7fd4f2ba1657576213f254a/users/d89c4aa62b3f4247a2d76011f7b2d12c/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:15.681756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772a29a-0ebb-4e6c-bf78-32f0ce926844 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.689433 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:15.720589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26decf3-7887-4e4b-8721-6948d4a84b5e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.742356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772a29a-0ebb-4e6c-bf78-32f0ce926844 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.747903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239ec9b-5488-4eb9-a154-d88c70693244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.748541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ec9b-5488-4eb9-a154-d88c70693244 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.748830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ec9b-5488-4eb9-a154-d88c70693244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.749102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ec9b-5488-4eb9-a154-d88c70693244 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.749722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239ec9b-5488-4eb9-a154-d88c70693244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.751807 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239ec9b-5488-4eb9-a154-d88c70693244 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.775259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd335a3-338d-4ebf-9223-e684c7d5ce1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.776063 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd335a3-338d-4ebf-9223-e684c7d5ce1b admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.776341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd335a3-338d-4ebf-9223-e684c7d5ce1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.776967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd335a3-338d-4ebf-9223-e684c7d5ce1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.777629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd335a3-338d-4ebf-9223-e684c7d5ce1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.779529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd335a3-338d-4ebf-9223-e684c7d5ce1b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.827242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 785/1575] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:15 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:15.828220 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239ec9b-5488-4eb9-a154-d88c70693244 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.837502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239ec9b-5488-4eb9-a154-d88c70693244 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.842101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e545997-6d3d-4bc5-b31c-ed42ea1028fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.842666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e545997-6d3d-4bc5-b31c-ed42ea1028fa admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.843025 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e545997-6d3d-4bc5-b31c-ed42ea1028fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.843294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e545997-6d3d-4bc5-b31c-ed42ea1028fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.843936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e545997-6d3d-4bc5-b31c-ed42ea1028fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.845541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e545997-6d3d-4bc5-b31c-ed42ea1028fa admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.853969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 791/1576] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:15 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:15.853969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd335a3-338d-4ebf-9223-e684c7d5ce1b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.859517 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 786/1577] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:15.860569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e545997-6d3d-4bc5-b31c-ed42ea1028fa admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:15.870399 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e545997-6d3d-4bc5-b31c-ed42ea1028fa admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.916432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd335a3-338d-4ebf-9223-e684c7d5ce1b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:15.920972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf61ebd-ebca-4d5b-b6ff-f14d4de3696f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.921580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf61ebd-ebca-4d5b-b6ff-f14d4de3696f admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.921863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf61ebd-ebca-4d5b-b6ff-f14d4de3696f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.922131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf61ebd-ebca-4d5b-b6ff-f14d4de3696f admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.922746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf61ebd-ebca-4d5b-b6ff-f14d4de3696f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.924687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf61ebd-ebca-4d5b-b6ff-f14d4de3696f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:15.927283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa435cbd-5213-4d7a-ac00-0fd797272922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:15.927883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa435cbd-5213-4d7a-ac00-0fd797272922 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:15.928193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa435cbd-5213-4d7a-ac00-0fd797272922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:15.928474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa435cbd-5213-4d7a-ac00-0fd797272922 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:15.929091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa435cbd-5213-4d7a-ac00-0fd797272922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:15.931032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa435cbd-5213-4d7a-ac00-0fd797272922 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.057849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 792/1578] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:15 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:16.058668 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf61ebd-ebca-4d5b-b6ff-f14d4de3696f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.107608 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:16.122184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf61ebd-ebca-4d5b-b6ff-f14d4de3696f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.126978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd51972-c8b3-4fad-9672-d20b52b84963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.127601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd51972-c8b3-4fad-9672-d20b52b84963 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.127842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd51972-c8b3-4fad-9672-d20b52b84963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.128143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd51972-c8b3-4fad-9672-d20b52b84963 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.128828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd51972-c8b3-4fad-9672-d20b52b84963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.130752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd51972-c8b3-4fad-9672-d20b52b84963 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.131262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 787/1579] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:15 2022] POST /identity/v3/users => generated 287 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:16.132521 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa435cbd-5213-4d7a-ac00-0fd797272922 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.143188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa435cbd-5213-4d7a-ac00-0fd797272922 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.147901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aead23f-02ee-4973-997d-e233fcbd8d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.148527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aead23f-02ee-4973-997d-e233fcbd8d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.148808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aead23f-02ee-4973-997d-e233fcbd8d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.149083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aead23f-02ee-4973-997d-e233fcbd8d40 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.149791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aead23f-02ee-4973-997d-e233fcbd8d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.151727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aead23f-02ee-4973-997d-e233fcbd8d40 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.193227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 793/1580] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:16 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:16.194451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af09b4-d55c-4cb1-a030-e45a604fa15b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.194758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af09b4-d55c-4cb1-a030-e45a604fa15b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.195024 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af09b4-d55c-4cb1-a030-e45a604fa15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.214124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 788/1581] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:16 2022] POST /identity/v3/users => generated 302 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:16.214124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aead23f-02ee-4973-997d-e233fcbd8d40 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.221443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aead23f-02ee-4973-997d-e233fcbd8d40 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.223501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d7af09b4-d55c-4cb1-a030-e45a604fa15b None None] MFA Rules not processed for user `d89c4aa62b3f4247a2d76011f7b2d12c`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:16.224876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d7af09b4-d55c-4cb1-a030-e45a604fa15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:16.226155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d10b35-7047-4dfd-be95-ddb70c5d947b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.226742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d10b35-7047-4dfd-be95-ddb70c5d947b admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.227024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d10b35-7047-4dfd-be95-ddb70c5d947b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.227307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d10b35-7047-4dfd-be95-ddb70c5d947b admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.228166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d10b35-7047-4dfd-be95-ddb70c5d947b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.229753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d10b35-7047-4dfd-be95-ddb70c5d947b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.240642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 789/1582] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:16.241429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d10b35-7047-4dfd-be95-ddb70c5d947b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.252112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d10b35-7047-4dfd-be95-ddb70c5d947b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.257218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946455ff-b23f-48aa-b768-12abb3931df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.257812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946455ff-b23f-48aa-b768-12abb3931df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.258091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946455ff-b23f-48aa-b768-12abb3931df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.258371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946455ff-b23f-48aa-b768-12abb3931df5 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.259694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946455ff-b23f-48aa-b768-12abb3931df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.261667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946455ff-b23f-48aa-b768-12abb3931df5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.290803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 794/1583] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:16.291651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af09b4-d55c-4cb1-a030-e45a604fa15b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.301665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af09b4-d55c-4cb1-a030-e45a604fa15b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.309849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11506fb-f1d8-493e-9b8b-91291f9aaa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.309849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11506fb-f1d8-493e-9b8b-91291f9aaa7c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.309849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11506fb-f1d8-493e-9b8b-91291f9aaa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.309849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11506fb-f1d8-493e-9b8b-91291f9aaa7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.310240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11506fb-f1d8-493e-9b8b-91291f9aaa7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.310328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11506fb-f1d8-493e-9b8b-91291f9aaa7c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.320058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 795/1584] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:16.320821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11506fb-f1d8-493e-9b8b-91291f9aaa7c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.330514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11506fb-f1d8-493e-9b8b-91291f9aaa7c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.335133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db1ec6-0f4a-4e90-940a-2202731afa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.335716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db1ec6-0f4a-4e90-940a-2202731afa6a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.339105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 790/1585] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:16 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:16.339267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db1ec6-0f4a-4e90-940a-2202731afa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.339267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db1ec6-0f4a-4e90-940a-2202731afa6a admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.339480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946455ff-b23f-48aa-b768-12abb3931df5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.339573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db1ec6-0f4a-4e90-940a-2202731afa6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.339573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db1ec6-0f4a-4e90-940a-2202731afa6a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.346600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946455ff-b23f-48aa-b768-12abb3931df5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.351503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844c483-9cea-4e41-8d5b-7cfaf969185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.352118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844c483-9cea-4e41-8d5b-7cfaf969185a admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.352389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844c483-9cea-4e41-8d5b-7cfaf969185a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.353185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844c483-9cea-4e41-8d5b-7cfaf969185a admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.353955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844c483-9cea-4e41-8d5b-7cfaf969185a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.355530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844c483-9cea-4e41-8d5b-7cfaf969185a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.365461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 791/1586] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:16.366241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844c483-9cea-4e41-8d5b-7cfaf969185a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.376117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844c483-9cea-4e41-8d5b-7cfaf969185a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.380688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c15cb0-0e49-4836-8af0-83878e420295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.381267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c15cb0-0e49-4836-8af0-83878e420295 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.381566 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c15cb0-0e49-4836-8af0-83878e420295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.382075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c15cb0-0e49-4836-8af0-83878e420295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.382677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c15cb0-0e49-4836-8af0-83878e420295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.384291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c15cb0-0e49-4836-8af0-83878e420295 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.385488 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:16.394744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 792/1587] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:16.395500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c15cb0-0e49-4836-8af0-83878e420295 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.405089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c15cb0-0e49-4836-8af0-83878e420295 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.409675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e206ea-75dd-45e8-8bd3-0b0ee42f0dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.413976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e206ea-75dd-45e8-8bd3-0b0ee42f0dfc admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.413976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e206ea-75dd-45e8-8bd3-0b0ee42f0dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.413976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e206ea-75dd-45e8-8bd3-0b0ee42f0dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.413976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e206ea-75dd-45e8-8bd3-0b0ee42f0dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.413976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e206ea-75dd-45e8-8bd3-0b0ee42f0dfc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.418143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 796/1588] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:16 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:16.419184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db1ec6-0f4a-4e90-940a-2202731afa6a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.426254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 793/1589] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:16.426254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e206ea-75dd-45e8-8bd3-0b0ee42f0dfc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.429967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db1ec6-0f4a-4e90-940a-2202731afa6a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.434618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b854914-676d-4850-962e-eec4b5035df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.435205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e206ea-75dd-45e8-8bd3-0b0ee42f0dfc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.435352 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b854914-676d-4850-962e-eec4b5035df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.435621 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b854914-676d-4850-962e-eec4b5035df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.435899 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b854914-676d-4850-962e-eec4b5035df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.436579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b854914-676d-4850-962e-eec4b5035df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.438181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b854914-676d-4850-962e-eec4b5035df0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.441016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a958f-9239-4346-a2a1-78677cb6630c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.442756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a958f-9239-4346-a2a1-78677cb6630c admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.442756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a958f-9239-4346-a2a1-78677cb6630c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.442756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a958f-9239-4346-a2a1-78677cb6630c admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.443013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a958f-9239-4346-a2a1-78677cb6630c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.444832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a958f-9239-4346-a2a1-78677cb6630c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.452837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 797/1590] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:16.452837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b854914-676d-4850-962e-eec4b5035df0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.459874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b854914-676d-4850-962e-eec4b5035df0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.465281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef6d50-d611-43d4-9820-9986ce41c8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.470641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef6d50-d611-43d4-9820-9986ce41c8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.470641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef6d50-d611-43d4-9820-9986ce41c8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.470641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef6d50-d611-43d4-9820-9986ce41c8e6 admin admin] PATH_INFO: `/v3/projects/6cd3d94550bd467f8a88d75ec1208be1/users/dfd02222f0b14a588b51a792b02ca852/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.480765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef6d50-d611-43d4-9820-9986ce41c8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cd3d94550bd467f8a88d75ec1208be1, user_id=dfd02222f0b14a588b51a792b02ca852, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.481791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-95ef6d50-d611-43d4-9820-9986ce41c8e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:37:16.481791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Aug 07 22:37:16.481791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: Aug 07 22:37:16.483734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef6d50-d611-43d4-9820-9986ce41c8e6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.577932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 794/1591] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:16 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:16.578771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a958f-9239-4346-a2a1-78677cb6630c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.588759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 798/1592] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:16 2022] PUT /identity/v3/projects/6cd3d94550bd467f8a88d75ec1208be1/users/dfd02222f0b14a588b51a792b02ca852/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:16.589972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef6d50-d611-43d4-9820-9986ce41c8e6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.644331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a958f-9239-4346-a2a1-78677cb6630c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.654261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef6d50-d611-43d4-9820-9986ce41c8e6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.658997 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f10b9-ed1f-46f8-bb7f-85e62038014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.659599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f10b9-ed1f-46f8-bb7f-85e62038014e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.659881 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f10b9-ed1f-46f8-bb7f-85e62038014e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.660187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f10b9-ed1f-46f8-bb7f-85e62038014e admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.660836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f10b9-ed1f-46f8-bb7f-85e62038014e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.662437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f10b9-ed1f-46f8-bb7f-85e62038014e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.664507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:16.674210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 799/1593] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:16.674816 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f10b9-ed1f-46f8-bb7f-85e62038014e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.685237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f10b9-ed1f-46f8-bb7f-85e62038014e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.689999 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606261-6dd3-47f7-bcf8-dfcdb8bc22df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.690569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606261-6dd3-47f7-bcf8-dfcdb8bc22df admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.690847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606261-6dd3-47f7-bcf8-dfcdb8bc22df admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.691129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606261-6dd3-47f7-bcf8-dfcdb8bc22df admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.691754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a606261-6dd3-47f7-bcf8-dfcdb8bc22df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.693339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a606261-6dd3-47f7-bcf8-dfcdb8bc22df admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.701056 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c65f4f-b6cc-4129-a167-99faf86bc84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.701602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c65f4f-b6cc-4129-a167-99faf86bc84b admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.701884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c65f4f-b6cc-4129-a167-99faf86bc84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.702159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c65f4f-b6cc-4129-a167-99faf86bc84b admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.702807 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c65f4f-b6cc-4129-a167-99faf86bc84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.703520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 800/1594] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:16.704446 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c65f4f-b6cc-4129-a167-99faf86bc84b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.704671 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606261-6dd3-47f7-bcf8-dfcdb8bc22df admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.714832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606261-6dd3-47f7-bcf8-dfcdb8bc22df admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.718161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 795/1595] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:16.718161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c65f4f-b6cc-4129-a167-99faf86bc84b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.720954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd81ba-a0fb-47ac-87f9-94d6bf5f28cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.721547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd81ba-a0fb-47ac-87f9-94d6bf5f28cf admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.721823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd81ba-a0fb-47ac-87f9-94d6bf5f28cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.722092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd81ba-a0fb-47ac-87f9-94d6bf5f28cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.722724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd81ba-a0fb-47ac-87f9-94d6bf5f28cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.724628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd81ba-a0fb-47ac-87f9-94d6bf5f28cf admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.730253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c65f4f-b6cc-4129-a167-99faf86bc84b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.734863 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f374fc1-ae6f-423e-ae3e-0262fb0e7b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.735472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f374fc1-ae6f-423e-ae3e-0262fb0e7b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.735734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f374fc1-ae6f-423e-ae3e-0262fb0e7b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.736002 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f374fc1-ae6f-423e-ae3e-0262fb0e7b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.736658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f374fc1-ae6f-423e-ae3e-0262fb0e7b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.737644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 801/1596] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:16.738229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f374fc1-ae6f-423e-ae3e-0262fb0e7b1a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.738427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd81ba-a0fb-47ac-87f9-94d6bf5f28cf admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.747736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd81ba-a0fb-47ac-87f9-94d6bf5f28cf admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.753055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 796/1597] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:16.753055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f374fc1-ae6f-423e-ae3e-0262fb0e7b1a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.753276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562f25a-8ebc-48e8-9981-f1892ffa3d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.753276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562f25a-8ebc-48e8-9981-f1892ffa3d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.753452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562f25a-8ebc-48e8-9981-f1892ffa3d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.753709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562f25a-8ebc-48e8-9981-f1892ffa3d6a admin admin] PATH_INFO: `/v3/projects/6fcdd19a618d41f9b498017b0197f4fd/users/ac8114f039494058804c49d95fe14176/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.760576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f374fc1-ae6f-423e-ae3e-0262fb0e7b1a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.765204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fe493-d9fe-4b9e-bee0-b93ffb1590d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.765793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fe493-d9fe-4b9e-bee0-b93ffb1590d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.766098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fe493-d9fe-4b9e-bee0-b93ffb1590d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.766376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fe493-d9fe-4b9e-bee0-b93ffb1590d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.767017 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fe493-d9fe-4b9e-bee0-b93ffb1590d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.768963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fe493-d9fe-4b9e-bee0-b93ffb1590d5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.769162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562f25a-8ebc-48e8-9981-f1892ffa3d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fcdd19a618d41f9b498017b0197f4fd, user_id=ac8114f039494058804c49d95fe14176, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.771819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562f25a-8ebc-48e8-9981-f1892ffa3d6a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.850256 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 802/1598] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:16 2022] PUT /identity/v3/projects/6fcdd19a618d41f9b498017b0197f4fd/users/ac8114f039494058804c49d95fe14176/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:16.852220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08ee9e-81c9-4731-8a08-5e2119d4c11f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.853088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08ee9e-81c9-4731-8a08-5e2119d4c11f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.853550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08ee9e-81c9-4731-8a08-5e2119d4c11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.869885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 797/1599] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:16 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:16.869885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fe493-d9fe-4b9e-bee0-b93ffb1590d5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.871902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9b08ee9e-81c9-4731-8a08-5e2119d4c11f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:16.874527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9b08ee9e-81c9-4731-8a08-5e2119d4c11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:16.879685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fe493-d9fe-4b9e-bee0-b93ffb1590d5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.912256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:16.936062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 803/1600] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:16.936430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863b63b-a5ba-4966-bd1e-7f5c91f7ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.936809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08ee9e-81c9-4731-8a08-5e2119d4c11f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:16.937153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863b63b-a5ba-4966-bd1e-7f5c91f7ce0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.937547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863b63b-a5ba-4966-bd1e-7f5c91f7ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.937893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863b63b-a5ba-4966-bd1e-7f5c91f7ce0f admin admin] PATH_INFO: `/v3/projects/c407485082104169b5ded0ca119f1577/users/5fed0c0e628e4468ae88225445693d2c/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.950471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08ee9e-81c9-4731-8a08-5e2119d4c11f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:16.952813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70245801-9116-4dbd-abca-45ad2fb9d79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:16.953466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70245801-9116-4dbd-abca-45ad2fb9d79d admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:16.953738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70245801-9116-4dbd-abca-45ad2fb9d79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:16.953844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863b63b-a5ba-4966-bd1e-7f5c91f7ce0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c407485082104169b5ded0ca119f1577, user_id=5fed0c0e628e4468ae88225445693d2c, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.954058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70245801-9116-4dbd-abca-45ad2fb9d79d admin admin] PATH_INFO: `/v3/projects/1a78ad9d7151473798fbefa330678250/users/966045fbe69f43b4b1a825da3ad571b9/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:16.956514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863b63b-a5ba-4966-bd1e-7f5c91f7ce0f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:16.969473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70245801-9116-4dbd-abca-45ad2fb9d79d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a78ad9d7151473798fbefa330678250, user_id=966045fbe69f43b4b1a825da3ad571b9, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:16.972262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70245801-9116-4dbd-abca-45ad2fb9d79d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:17.019075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 798/1601] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:16 2022] PUT /identity/v3/projects/c407485082104169b5ded0ca119f1577/users/5fed0c0e628e4468ae88225445693d2c/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:17.019914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863b63b-a5ba-4966-bd1e-7f5c91f7ce0f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:17.055167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 804/1602] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:16 2022] PUT /identity/v3/projects/1a78ad9d7151473798fbefa330678250/users/966045fbe69f43b4b1a825da3ad571b9/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:17.056677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70245801-9116-4dbd-abca-45ad2fb9d79d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:17.082236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863b63b-a5ba-4966-bd1e-7f5c91f7ce0f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:17.121511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70245801-9116-4dbd-abca-45ad2fb9d79d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:17.126215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565339d6-f42d-4dc4-8dac-e8d3abcda1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:17.126819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565339d6-f42d-4dc4-8dac-e8d3abcda1be admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.127101 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565339d6-f42d-4dc4-8dac-e8d3abcda1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.127634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565339d6-f42d-4dc4-8dac-e8d3abcda1be admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.128008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565339d6-f42d-4dc4-8dac-e8d3abcda1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:17.129667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565339d6-f42d-4dc4-8dac-e8d3abcda1be admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:17.138097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d1ee8-6796-4a44-b9f5-4656a04d1c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:17.138729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d1ee8-6796-4a44-b9f5-4656a04d1c09 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.139007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d1ee8-6796-4a44-b9f5-4656a04d1c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.139277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d1ee8-6796-4a44-b9f5-4656a04d1c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.139913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d1ee8-6796-4a44-b9f5-4656a04d1c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:17.140903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 805/1603] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:17.141561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d1ee8-6796-4a44-b9f5-4656a04d1c09 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:17.141742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565339d6-f42d-4dc4-8dac-e8d3abcda1be admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:17.152136 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565339d6-f42d-4dc4-8dac-e8d3abcda1be admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:17.156706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 799/1604] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:17.156706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7bc46-cd87-44c7-99ac-e1f85e04aa14 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.156706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7bc46-cd87-44c7-99ac-e1f85e04aa14 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.156706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7bc46-cd87-44c7-99ac-e1f85e04aa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.160479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb59e8a-51b4-42ac-9f0d-041463836473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:17.161127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb59e8a-51b4-42ac-9f0d-041463836473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.161408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb59e8a-51b4-42ac-9f0d-041463836473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.161689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb59e8a-51b4-42ac-9f0d-041463836473 admin admin] PATH_INFO: `/v3/projects/c1c8b88e7f944005a03d220e4e70c9cb/users/b79ec2edd94846ab973e8ece6b78aa89/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.177228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb59e8a-51b4-42ac-9f0d-041463836473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c8b88e7f944005a03d220e4e70c9cb, user_id=b79ec2edd94846ab973e8ece6b78aa89, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:17.179888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb59e8a-51b4-42ac-9f0d-041463836473 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:17.181489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6ff7bc46-cd87-44c7-99ac-e1f85e04aa14 None None] MFA Rules not processed for user `ac8114f039494058804c49d95fe14176`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:17.182704 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6ff7bc46-cd87-44c7-99ac-e1f85e04aa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:17.236294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:17.246675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:17.251556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 800/1605] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:17.251556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7bc46-cd87-44c7-99ac-e1f85e04aa14 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:17.252332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 806/1606] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:17 2022] PUT /identity/v3/projects/c1c8b88e7f944005a03d220e4e70c9cb/users/b79ec2edd94846ab973e8ece6b78aa89/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:17.257089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb59e8a-51b4-42ac-9f0d-041463836473 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:17.258932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7bc46-cd87-44c7-99ac-e1f85e04aa14 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:17.315797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb59e8a-51b4-42ac-9f0d-041463836473 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:17.319767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c3bbd-e946-4d87-8646-907e3b0fe0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:17.319949 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b71b2-05d5-4e61-ae5c-03abc413415b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:17.320472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b71b2-05d5-4e61-ae5c-03abc413415b service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.320715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c3bbd-e946-4d87-8646-907e3b0fe0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.320843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b71b2-05d5-4e61-ae5c-03abc413415b service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.321000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c3bbd-e946-4d87-8646-907e3b0fe0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.321108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b71b2-05d5-4e61-ae5c-03abc413415b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.321411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c3bbd-e946-4d87-8646-907e3b0fe0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.322501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c3bbd-e946-4d87-8646-907e3b0fe0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:17.324159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c3bbd-e946-4d87-8646-907e3b0fe0d3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:17.330401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b71b2-05d5-4e61-ae5c-03abc413415b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:17.331836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b71b2-05d5-4e61-ae5c-03abc413415b service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:17.334123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 801/1607] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:17.335327 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce8401-703c-4cbe-a6f1-c57eb25aa47f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.335618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce8401-703c-4cbe-a6f1-c57eb25aa47f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.335894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce8401-703c-4cbe-a6f1-c57eb25aa47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.342926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 807/1608] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:17.348579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2d8c3-3bb5-4b0a-ae2f-e761b38553ba None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.348579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2d8c3-3bb5-4b0a-ae2f-e761b38553ba None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.348579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2d8c3-3bb5-4b0a-ae2f-e761b38553ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.365652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f5ce8401-703c-4cbe-a6f1-c57eb25aa47f None None] MFA Rules not processed for user `5fed0c0e628e4468ae88225445693d2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:17.367008 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f5ce8401-703c-4cbe-a6f1-c57eb25aa47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:17.368466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d3b2d8c3-3bb5-4b0a-ae2f-e761b38553ba None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:17.369661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d3b2d8c3-3bb5-4b0a-ae2f-e761b38553ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:17.378826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 808/1609] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:17.379360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2d8c3-3bb5-4b0a-ae2f-e761b38553ba None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:17.389537 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2d8c3-3bb5-4b0a-ae2f-e761b38553ba None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:17.394927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb6a5d-f2c0-437c-99d9-f551b6e44e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:17.395519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb6a5d-f2c0-437c-99d9-f551b6e44e67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.395839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb6a5d-f2c0-437c-99d9-f551b6e44e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.396147 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb6a5d-f2c0-437c-99d9-f551b6e44e67 admin admin] PATH_INFO: `/v3/projects/bf8ae1dcb7bb4d58b383f8e992528f62/users/c461b1dffb1a412b9fb69bb3b2c4a5bd/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.413104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb6a5d-f2c0-437c-99d9-f551b6e44e67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf8ae1dcb7bb4d58b383f8e992528f62, user_id=c461b1dffb1a412b9fb69bb3b2c4a5bd, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:17.416722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb6a5d-f2c0-437c-99d9-f551b6e44e67 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:17.441495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 802/1610] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:17.445851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce8401-703c-4cbe-a6f1-c57eb25aa47f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:17.454220 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce8401-703c-4cbe-a6f1-c57eb25aa47f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:17.459052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe68d82-00f0-4e84-9454-f11cf89162a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:17.459641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe68d82-00f0-4e84-9454-f11cf89162a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.459922 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe68d82-00f0-4e84-9454-f11cf89162a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.460226 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe68d82-00f0-4e84-9454-f11cf89162a6 admin admin] PATH_INFO: `/v3/projects/3f7ea2694e7641088d957060a1001d86/users/05ef5ec6afb84c4abc0cb4862f041dd1/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.476214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe68d82-00f0-4e84-9454-f11cf89162a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f7ea2694e7641088d957060a1001d86, user_id=05ef5ec6afb84c4abc0cb4862f041dd1, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:17.479790 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe68d82-00f0-4e84-9454-f11cf89162a6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:17.486188 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 809/1611] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:17 2022] PUT /identity/v3/projects/bf8ae1dcb7bb4d58b383f8e992528f62/users/c461b1dffb1a412b9fb69bb3b2c4a5bd/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:17.487452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b1178-0bf4-4b20-b53c-f8c6539da01e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.487740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b1178-0bf4-4b20-b53c-f8c6539da01e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.488035 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b1178-0bf4-4b20-b53c-f8c6539da01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.507622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bb3b1178-0bf4-4b20-b53c-f8c6539da01e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:17.509197 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bb3b1178-0bf4-4b20-b53c-f8c6539da01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:17.543202 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 803/1612] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:17 2022] PUT /identity/v3/projects/3f7ea2694e7641088d957060a1001d86/users/05ef5ec6afb84c4abc0cb4862f041dd1/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:17.544837 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800617e-daa7-4aca-953a-199eddf78e5e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.545222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800617e-daa7-4aca-953a-199eddf78e5e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.545591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800617e-daa7-4aca-953a-199eddf78e5e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.547131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 804/1613] 10.222.0.30 () {54 vars in 962 bytes} [Sun Aug 7 22:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:37:17.547895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800617e-daa7-4aca-953a-199eddf78e5e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:17.568237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 810/1614] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:17.569136 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b1178-0bf4-4b20-b53c-f8c6539da01e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:17.632369 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:17.638415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800617e-daa7-4aca-953a-199eddf78e5e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:17.641749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b1178-0bf4-4b20-b53c-f8c6539da01e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:17.643284 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce0ef3-2b2a-45cc-899c-4f0983bac0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:17.644335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce0ef3-2b2a-45cc-899c-4f0983bac0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.644615 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce0ef3-2b2a-45cc-899c-4f0983bac0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.644899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce0ef3-2b2a-45cc-899c-4f0983bac0d2 admin admin] PATH_INFO: `/v3/projects/e6a6be36471846218b7cdd351274d071/users/c45bb9bee57a4fe9a99643dca9565f1e/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.661108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce0ef3-2b2a-45cc-899c-4f0983bac0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a6be36471846218b7cdd351274d071, user_id=c45bb9bee57a4fe9a99643dca9565f1e, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:17.663853 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce0ef3-2b2a-45cc-899c-4f0983bac0d2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:17.671407 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:17.694242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630074c-7641-45b2-b7c0-d31e481f47b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:17.694852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630074c-7641-45b2-b7c0-d31e481f47b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.695144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630074c-7641-45b2-b7c0-d31e481f47b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.695396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630074c-7641-45b2-b7c0-d31e481f47b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.704664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630074c-7641-45b2-b7c0-d31e481f47b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:17.706128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630074c-7641-45b2-b7c0-d31e481f47b3 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:17.717410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 811/1615] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:17.717410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630074c-7641-45b2-b7c0-d31e481f47b3 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:17.725401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630074c-7641-45b2-b7c0-d31e481f47b3 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:17.730968 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4a331-4a16-4c10-8844-34efa1fbd8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:17.731549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a331-4a16-4c10-8844-34efa1fbd8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.731824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a331-4a16-4c10-8844-34efa1fbd8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.732391 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a331-4a16-4c10-8844-34efa1fbd8b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.732732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4a331-4a16-4c10-8844-34efa1fbd8b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:17.734637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4a331-4a16-4c10-8844-34efa1fbd8b8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:17.742655 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 805/1616] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:17 2022] PUT /identity/v3/projects/e6a6be36471846218b7cdd351274d071/users/c45bb9bee57a4fe9a99643dca9565f1e/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:17.744149 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21424b73-afe7-477b-b3ef-9814400c5de8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.746368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21424b73-afe7-477b-b3ef-9814400c5de8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.746368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21424b73-afe7-477b-b3ef-9814400c5de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.773396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-21424b73-afe7-477b-b3ef-9814400c5de8 None None] MFA Rules not processed for user `c461b1dffb1a412b9fb69bb3b2c4a5bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:17.775270 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-21424b73-afe7-477b-b3ef-9814400c5de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:17.841789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 806/1617] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:17.844910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b5f4-9ce7-4099-92e8-53a44b28c481 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.847473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b5f4-9ce7-4099-92e8-53a44b28c481 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.848710 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b5f4-9ce7-4099-92e8-53a44b28c481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.857318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 812/1618] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:17 2022] POST /identity/v3/projects => generated 371 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:17.858979 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a758de-1357-49b0-904d-3433bb5c3e25 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:17.859240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a758de-1357-49b0-904d-3433bb5c3e25 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:17.859867 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a758de-1357-49b0-904d-3433bb5c3e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:17.874227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f3a5b5f4-9ce7-4099-92e8-53a44b28c481 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:17.875561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f3a5b5f4-9ce7-4099-92e8-53a44b28c481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:17.927391 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-83a758de-1357-49b0-904d-3433bb5c3e25 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:17.928837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-83a758de-1357-49b0-904d-3433bb5c3e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:17.965100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 807/1619] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:17.966694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5b5f4-9ce7-4099-92e8-53a44b28c481 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:17.969270 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f3a5b5f4-9ce7-4099-92e8-53a44b28c481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:18.025214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 813/1620] 10.222.0.30 () {58 vars in 1059 bytes} [Sun Aug 7 22:37:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:18.026557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a758de-1357-49b0-904d-3433bb5c3e25 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:18.036519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5b5f4-9ce7-4099-92e8-53a44b28c481 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:18.043753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c2691-2e37-4173-bf54-5a49b35a2e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:18.044709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c2691-2e37-4173-bf54-5a49b35a2e3c service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.048935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a758de-1357-49b0-904d-3433bb5c3e25 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:18.049066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c2691-2e37-4173-bf54-5a49b35a2e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.049066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c2691-2e37-4173-bf54-5a49b35a2e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.053144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7f503-40de-4791-88c4-a55208b54bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:18.053809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7f503-40de-4791-88c4-a55208b54bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.054090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7f503-40de-4791-88c4-a55208b54bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.054576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7f503-40de-4791-88c4-a55208b54bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.055250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7f503-40de-4791-88c4-a55208b54bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:18.057103 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7f503-40de-4791-88c4-a55208b54bb3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:18.062387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c2691-2e37-4173-bf54-5a49b35a2e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:18.064962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c2691-2e37-4173-bf54-5a49b35a2e3c service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:18.068818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 814/1621] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:18.069802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7f503-40de-4791-88c4-a55208b54bb3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:18.079999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 808/1622] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:18.081226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7f503-40de-4791-88c4-a55208b54bb3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:18.081778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c004dc-490e-4d31-b775-eee2d02de43e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.082143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c004dc-490e-4d31-b775-eee2d02de43e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.082494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c004dc-490e-4d31-b775-eee2d02de43e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.086451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f065009-f9c7-49cd-b5cc-4b1e247fcd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:18.087126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f065009-f9c7-49cd-b5cc-4b1e247fcd86 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.087410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f065009-f9c7-49cd-b5cc-4b1e247fcd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.087874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f065009-f9c7-49cd-b5cc-4b1e247fcd86 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.088585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f065009-f9c7-49cd-b5cc-4b1e247fcd86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:18.090597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f065009-f9c7-49cd-b5cc-4b1e247fcd86 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:18.091800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 809/1623] 10.222.0.30 () {54 vars in 962 bytes} [Sun Aug 7 22:37:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:37:18.092861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c004dc-490e-4d31-b775-eee2d02de43e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:18.108238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c004dc-490e-4d31-b775-eee2d02de43e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:18.113340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5b684-8c3d-4dbd-9eea-ba241b8edf9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:18.115295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5b684-8c3d-4dbd-9eea-ba241b8edf9b service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.115594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5b684-8c3d-4dbd-9eea-ba241b8edf9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.115938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5b684-8c3d-4dbd-9eea-ba241b8edf9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.130793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5b684-8c3d-4dbd-9eea-ba241b8edf9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:18.132886 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5b684-8c3d-4dbd-9eea-ba241b8edf9b service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:18.140843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:18.151844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 810/1624] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:18.153056 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5b684-8c3d-4dbd-9eea-ba241b8edf9b service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:18.167990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5b684-8c3d-4dbd-9eea-ba241b8edf9b service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:18.173979 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319fa73-58c6-4d68-8840-049991a2d99b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:18.175158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319fa73-58c6-4d68-8840-049991a2d99b service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.175445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319fa73-58c6-4d68-8840-049991a2d99b service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.175726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319fa73-58c6-4d68-8840-049991a2d99b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.197885 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 815/1625] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:18 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:18.197885 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f065009-f9c7-49cd-b5cc-4b1e247fcd86 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:18.208483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f065009-f9c7-49cd-b5cc-4b1e247fcd86 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:18.216092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc673ad-9925-4d80-a1f7-7624cf4e930e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:18.216769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc673ad-9925-4d80-a1f7-7624cf4e930e admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.217091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc673ad-9925-4d80-a1f7-7624cf4e930e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.217355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc673ad-9925-4d80-a1f7-7624cf4e930e admin admin] PATH_INFO: `/v3/projects/e6a6be36471846218b7cdd351274d071/users/c45bb9bee57a4fe9a99643dca9565f1e/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.219373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc673ad-9925-4d80-a1f7-7624cf4e930e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a6be36471846218b7cdd351274d071, user_id=c45bb9bee57a4fe9a99643dca9565f1e, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:18.222253 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc673ad-9925-4d80-a1f7-7624cf4e930e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:18.235493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 816/1626] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:18 2022] PUT /identity/v3/projects/e6a6be36471846218b7cdd351274d071/users/c45bb9bee57a4fe9a99643dca9565f1e/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:18.236796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1d39b-b426-4d53-8405-922eb7b544e7 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.237051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1d39b-b426-4d53-8405-922eb7b544e7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.237354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1d39b-b426-4d53-8405-922eb7b544e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.243045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319fa73-58c6-4d68-8840-049991a2d99b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:18.245792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319fa73-58c6-4d68-8840-049991a2d99b service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:18.257799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 811/1627] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:18.258004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6db1d39b-b426-4d53-8405-922eb7b544e7 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:18.258622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319fa73-58c6-4d68-8840-049991a2d99b service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:18.259203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6db1d39b-b426-4d53-8405-922eb7b544e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:18.269173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319fa73-58c6-4d68-8840-049991a2d99b service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:18.314869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:18.345527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 817/1628] 10.222.0.30 () {58 vars in 1059 bytes} [Sun Aug 7 22:37:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:18.347188 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b02868-4a41-411c-8e63-6350009b9ac0 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.347721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b02868-4a41-411c-8e63-6350009b9ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.348452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b02868-4a41-411c-8e63-6350009b9ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.392237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-78b02868-4a41-411c-8e63-6350009b9ac0 None None] MFA Rules not processed for user `c45bb9bee57a4fe9a99643dca9565f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:18.394918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a96ba7-53c2-4665-9cb9-6b5f5ce16e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:18.395541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a96ba7-53c2-4665-9cb9-6b5f5ce16e1c admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.395947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-78b02868-4a41-411c-8e63-6350009b9ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:18.396100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a96ba7-53c2-4665-9cb9-6b5f5ce16e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.396296 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a96ba7-53c2-4665-9cb9-6b5f5ce16e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.396959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a96ba7-53c2-4665-9cb9-6b5f5ce16e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:18.398626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a96ba7-53c2-4665-9cb9-6b5f5ce16e1c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:18.417887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 812/1629] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:18.420783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a96ba7-53c2-4665-9cb9-6b5f5ce16e1c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:18.449514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a96ba7-53c2-4665-9cb9-6b5f5ce16e1c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:18.464241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdcfb1-5e4a-4c98-89af-f0189e1d1cba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:18.465994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdcfb1-5e4a-4c98-89af-f0189e1d1cba service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.468446 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdcfb1-5e4a-4c98-89af-f0189e1d1cba service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.468446 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdcfb1-5e4a-4c98-89af-f0189e1d1cba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.536308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 818/1630] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:18.536308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b02868-4a41-411c-8e63-6350009b9ac0 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:18.560772 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b02868-4a41-411c-8e63-6350009b9ac0 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:18.574779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ed7a6-1a8b-4440-b755-7dde126e5931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:18.575666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ed7a6-1a8b-4440-b755-7dde126e5931 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.576151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ed7a6-1a8b-4440-b755-7dde126e5931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.576575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ed7a6-1a8b-4440-b755-7dde126e5931 admin admin] PATH_INFO: `/v3/projects/d4a459882af2438a8c9875e22905b605/users/280ce449e404404f9f20be0636bacbf8/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.608392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ed7a6-1a8b-4440-b755-7dde126e5931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4a459882af2438a8c9875e22905b605, user_id=280ce449e404404f9f20be0636bacbf8, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:18.613052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ed7a6-1a8b-4440-b755-7dde126e5931 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:18.617279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdcfb1-5e4a-4c98-89af-f0189e1d1cba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:18.620143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdcfb1-5e4a-4c98-89af-f0189e1d1cba service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:18.635084 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 813/1631] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:18.637076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdcfb1-5e4a-4c98-89af-f0189e1d1cba service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:18.678908 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:18.708863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 819/1632] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:18 2022] PUT /identity/v3/projects/d4a459882af2438a8c9875e22905b605/users/280ce449e404404f9f20be0636bacbf8/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:18.710756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fced2-531b-4434-a6f1-f79c23cbcc05 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.711697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fced2-531b-4434-a6f1-f79c23cbcc05 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.712793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fced2-531b-4434-a6f1-f79c23cbcc05 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.715556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 820/1633] 10.222.0.30 () {54 vars in 912 bytes} [Sun Aug 7 22:37:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:37:18.726580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fced2-531b-4434-a6f1-f79c23cbcc05 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:18.732577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdcfb1-5e4a-4c98-89af-f0189e1d1cba service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:18.804729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c5817-c987-47f1-8916-677eda4031d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:18.806337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c5817-c987-47f1-8916-677eda4031d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.806754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c5817-c987-47f1-8916-677eda4031d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.807122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c5817-c987-47f1-8916-677eda4031d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.872682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fced2-531b-4434-a6f1-f79c23cbcc05 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:18.885086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c5817-c987-47f1-8916-677eda4031d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:18.888515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c5817-c987-47f1-8916-677eda4031d7 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:18.891663 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833a8df-8299-45f3-86ce-657f2135f70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:18.892326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833a8df-8299-45f3-86ce-657f2135f70d admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.892605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833a8df-8299-45f3-86ce-657f2135f70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.892883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833a8df-8299-45f3-86ce-657f2135f70d admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.893544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833a8df-8299-45f3-86ce-657f2135f70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:18.895308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833a8df-8299-45f3-86ce-657f2135f70d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:18.900324 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 814/1634] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:18.901876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2cfea-d0a4-4ae0-a7e4-48261c1f41c4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.902297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2cfea-d0a4-4ae0-a7e4-48261c1f41c4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.902873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2cfea-d0a4-4ae0-a7e4-48261c1f41c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.935287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2db2cfea-d0a4-4ae0-a7e4-48261c1f41c4 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:18.936666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2db2cfea-d0a4-4ae0-a7e4-48261c1f41c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:18.952693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 821/1635] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:18.954468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13066177-5f89-45d8-9bc8-7706468aaa54 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:18.963048 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13066177-5f89-45d8-9bc8-7706468aaa54 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:18.963553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13066177-5f89-45d8-9bc8-7706468aaa54 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:18.965215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 822/1636] 10.222.0.30 () {54 vars in 912 bytes} [Sun Aug 7 22:37:18 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:37:18.966119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13066177-5f89-45d8-9bc8-7706468aaa54 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:19.001794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13066177-5f89-45d8-9bc8-7706468aaa54 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:19.014830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cda9c-ed9b-4460-a75c-c865abae44e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:19.015525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cda9c-ed9b-4460-a75c-c865abae44e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:19.015904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cda9c-ed9b-4460-a75c-c865abae44e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:19.016210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cda9c-ed9b-4460-a75c-c865abae44e6 admin admin] PATH_INFO: `/v3/projects/d4a459882af2438a8c9875e22905b605/users/280ce449e404404f9f20be0636bacbf8/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:19.019204 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cda9c-ed9b-4460-a75c-c865abae44e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4a459882af2438a8c9875e22905b605, user_id=280ce449e404404f9f20be0636bacbf8, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:19.022442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cda9c-ed9b-4460-a75c-c865abae44e6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:19.039114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 823/1637] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:18 2022] PUT /identity/v3/projects/d4a459882af2438a8c9875e22905b605/users/280ce449e404404f9f20be0636bacbf8/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:19.040631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e0b74-d28f-47df-8c68-c5feb8544829 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:19.040996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e0b74-d28f-47df-8c68-c5feb8544829 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:19.041269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e0b74-d28f-47df-8c68-c5feb8544829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:19.071301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-dc3e0b74-d28f-47df-8c68-c5feb8544829 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:19.092254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dc3e0b74-d28f-47df-8c68-c5feb8544829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:19.118683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:19.122793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 815/1638] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:37:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:19.132733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca8758-f6bc-4f3f-9053-544fb16d1aec None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:19.133094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca8758-f6bc-4f3f-9053-544fb16d1aec None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:19.133445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca8758-f6bc-4f3f-9053-544fb16d1aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:19.190222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-98ca8758-f6bc-4f3f-9053-544fb16d1aec None None] MFA Rules not processed for user `280ce449e404404f9f20be0636bacbf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:19.211237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-98ca8758-f6bc-4f3f-9053-544fb16d1aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:19.226690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 824/1639] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:37:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:19.227463 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e0b74-d28f-47df-8c68-c5feb8544829 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:19.246426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:19.294267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e0b74-d28f-47df-8c68-c5feb8544829 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:19.298437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99f1ae-7046-4a99-958e-203f1d29eaa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:19.299129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99f1ae-7046-4a99-958e-203f1d29eaa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:19.299472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99f1ae-7046-4a99-958e-203f1d29eaa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:19.299932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99f1ae-7046-4a99-958e-203f1d29eaa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:19.312533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99f1ae-7046-4a99-958e-203f1d29eaa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:19.314018 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99f1ae-7046-4a99-958e-203f1d29eaa0 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:19.318949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 816/1640] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:19.318949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca8758-f6bc-4f3f-9053-544fb16d1aec None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:19.328350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 825/1641] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:19.340164 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca8758-f6bc-4f3f-9053-544fb16d1aec None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:19.344952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0f7f0-6184-4a25-acb5-8a5b70abac93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:19.345670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0f7f0-6184-4a25-acb5-8a5b70abac93 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:19.346224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0f7f0-6184-4a25-acb5-8a5b70abac93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:19.346602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0f7f0-6184-4a25-acb5-8a5b70abac93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:19.375394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0f7f0-6184-4a25-acb5-8a5b70abac93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:19.377310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0f7f0-6184-4a25-acb5-8a5b70abac93 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:19.396281 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 817/1642] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:19.548519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99f1ae-7046-4a99-958e-203f1d29eaa0 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:19.559604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99f1ae-7046-4a99-958e-203f1d29eaa0 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:19.564238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3e673-ea82-4f61-a401-aa3b27ba9cb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:19.564858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3e673-ea82-4f61-a401-aa3b27ba9cb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:19.565137 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3e673-ea82-4f61-a401-aa3b27ba9cb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:19.565410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3e673-ea82-4f61-a401-aa3b27ba9cb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:19.577648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3e673-ea82-4f61-a401-aa3b27ba9cb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:19.581399 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3e673-ea82-4f61-a401-aa3b27ba9cb1 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:19.601897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 826/1643] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:20.359726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0f7f0-6184-4a25-acb5-8a5b70abac93 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:20.385512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3e673-ea82-4f61-a401-aa3b27ba9cb1 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:20.406469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3e673-ea82-4f61-a401-aa3b27ba9cb1 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:20.425912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e03174-8785-4887-be07-920ef004ed57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:20.431393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e03174-8785-4887-be07-920ef004ed57 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:20.431393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e03174-8785-4887-be07-920ef004ed57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:20.431393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e03174-8785-4887-be07-920ef004ed57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:20.601180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e03174-8785-4887-be07-920ef004ed57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:20.605920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e03174-8785-4887-be07-920ef004ed57 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:20.609260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0f7f0-6184-4a25-acb5-8a5b70abac93 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:20.622125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a953c-5158-49b1-8b73-017855369d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:20.622948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a953c-5158-49b1-8b73-017855369d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:20.623338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a953c-5158-49b1-8b73-017855369d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:20.623721 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a953c-5158-49b1-8b73-017855369d9c admin admin] PATH_INFO: `/v3/users/ac8114f039494058804c49d95fe14176` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:20.627111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a953c-5158-49b1-8b73-017855369d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac8114f039494058804c49d95fe14176)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:20.629251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a953c-5158-49b1-8b73-017855369d9c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:20.641381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 827/1644] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:20.709690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6552b-34b0-4f13-a7e7-60f61c3c1957 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:20.710355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6552b-34b0-4f13-a7e7-60f61c3c1957 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:20.712389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6552b-34b0-4f13-a7e7-60f61c3c1957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:20.896286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-12e6552b-34b0-4f13-a7e7-60f61c3c1957 None None] MFA Rules not processed for user `b79ec2edd94846ab973e8ece6b78aa89`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:20.896286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-12e6552b-34b0-4f13-a7e7-60f61c3c1957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:20.896286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:20.896286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 828/1645] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:20.896286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6552b-34b0-4f13-a7e7-60f61c3c1957 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:20.896286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6552b-34b0-4f13-a7e7-60f61c3c1957 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:20.897472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f5b46-5a65-4541-aa3f-ca9fb6b50f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:20.897472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f5b46-5a65-4541-aa3f-ca9fb6b50f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:20.897472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f5b46-5a65-4541-aa3f-ca9fb6b50f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:20.897472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f5b46-5a65-4541-aa3f-ca9fb6b50f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:20.897815 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f5b46-5a65-4541-aa3f-ca9fb6b50f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:20.897815 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f5b46-5a65-4541-aa3f-ca9fb6b50f38 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:20.911723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b55a953c-5158-49b1-8b73-017855369d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac8114f039494058804c49d95fe14176'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:21.000357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b55a953c-5158-49b1-8b73-017855369d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac8114f039494058804c49d95fe14176'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:21.029585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 829/1646] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:20 2022] POST /identity/v3/projects => generated 405 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:21.038880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f5b46-5a65-4541-aa3f-ca9fb6b50f38 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:21.068087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b55a953c-5158-49b1-8b73-017855369d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac8114f039494058804c49d95fe14176'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:21.070047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b55a953c-5158-49b1-8b73-017855369d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac8114f039494058804c49d95fe14176'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:21.077657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b55a953c-5158-49b1-8b73-017855369d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac8114f039494058804c49d95fe14176'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:21.135300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 818/1647] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:37:20 2022] DELETE /identity/v3/users/ac8114f039494058804c49d95fe14176 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:21.139397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b33a5-3575-4cd7-9358-ed5cb7e2a7ce None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:21.139858 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b33a5-3575-4cd7-9358-ed5cb7e2a7ce None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:21.140249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b33a5-3575-4cd7-9358-ed5cb7e2a7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:21.190964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b92b33a5-3575-4cd7-9358-ed5cb7e2a7ce None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:21.191888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b92b33a5-3575-4cd7-9358-ed5cb7e2a7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:21.212286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:21.223555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f5b46-5a65-4541-aa3f-ca9fb6b50f38 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:21.309944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 819/1648] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:37:21.354576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b33a5-3575-4cd7-9358-ed5cb7e2a7ce None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:21.356159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b92b33a5-3575-4cd7-9358-ed5cb7e2a7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:21.373189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2a0a5-a12a-4ea8-98bc-9c70db75d913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:21.373981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2a0a5-a12a-4ea8-98bc-9c70db75d913 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:21.374269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2a0a5-a12a-4ea8-98bc-9c70db75d913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:21.374551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2a0a5-a12a-4ea8-98bc-9c70db75d913 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:21.375201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2a0a5-a12a-4ea8-98bc-9c70db75d913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:21.377202 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2a0a5-a12a-4ea8-98bc-9c70db75d913 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:21.475062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 830/1649] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:21 2022] POST /identity/v3/users => generated 309 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:21.482917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2a0a5-a12a-4ea8-98bc-9c70db75d913 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:21.485146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9dc2a0a5-a12a-4ea8-98bc-9c70db75d913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:21.544945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b33a5-3575-4cd7-9358-ed5cb7e2a7ce None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:21.551294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efb410-ac80-4e8e-a91e-6dbffba386e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:21.551936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb410-ac80-4e8e-a91e-6dbffba386e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:21.552259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb410-ac80-4e8e-a91e-6dbffba386e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:21.552533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb410-ac80-4e8e-a91e-6dbffba386e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:21.578922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2a0a5-a12a-4ea8-98bc-9c70db75d913 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:21.604552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44fa5d-22b3-4897-b834-b6cd4c3e79c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:21.605097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efb410-ac80-4e8e-a91e-6dbffba386e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:21.605198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44fa5d-22b3-4897-b834-b6cd4c3e79c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:21.605487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44fa5d-22b3-4897-b834-b6cd4c3e79c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:21.607557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44fa5d-22b3-4897-b834-b6cd4c3e79c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:21.608677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44fa5d-22b3-4897-b834-b6cd4c3e79c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:21.609455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efb410-ac80-4e8e-a91e-6dbffba386e8 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:21.615469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44fa5d-22b3-4897-b834-b6cd4c3e79c2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:21.647984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 820/1650] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:21.671010 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efb410-ac80-4e8e-a91e-6dbffba386e8 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:21.672282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 831/1651] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:21.688631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efb410-ac80-4e8e-a91e-6dbffba386e8 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:21.721993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd717c9-047b-4497-86d1-81bfc17261ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:21.722759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd717c9-047b-4497-86d1-81bfc17261ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:21.723112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd717c9-047b-4497-86d1-81bfc17261ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:21.723508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd717c9-047b-4497-86d1-81bfc17261ef admin admin] PATH_INFO: `/v3/projects/6feca6988aa74d0ca67f74e2b6a93796/users/d4aab2437e0a407aa21dd3062d0d72c5/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:21.767543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd717c9-047b-4497-86d1-81bfc17261ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6feca6988aa74d0ca67f74e2b6a93796, user_id=d4aab2437e0a407aa21dd3062d0d72c5, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:21.771068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd717c9-047b-4497-86d1-81bfc17261ef admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:21.907355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 821/1652] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:21 2022] PUT /identity/v3/projects/6feca6988aa74d0ca67f74e2b6a93796/users/d4aab2437e0a407aa21dd3062d0d72c5/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:37:21.911626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44fa5d-22b3-4897-b834-b6cd4c3e79c2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:21.930395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44fa5d-22b3-4897-b834-b6cd4c3e79c2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:21.998484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:22.047355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd717c9-047b-4497-86d1-81bfc17261ef admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:22.051930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9cd717c9-047b-4497-86d1-81bfc17261ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:22.079507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd43e4-b6a0-454b-bf46-1354a3203e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:22.079993 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd43e4-b6a0-454b-bf46-1354a3203e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:22.080293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd43e4-b6a0-454b-bf46-1354a3203e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:22.080567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd43e4-b6a0-454b-bf46-1354a3203e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:22.089773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd43e4-b6a0-454b-bf46-1354a3203e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:22.094308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd43e4-b6a0-454b-bf46-1354a3203e3f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:22.121307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 832/1653] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:22.123254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd43e4-b6a0-454b-bf46-1354a3203e3f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:22.143828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd43e4-b6a0-454b-bf46-1354a3203e3f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:22.158047 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c61371-0b11-4e81-9a7e-3d57c4ed8c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:22.158737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61371-0b11-4e81-9a7e-3d57c4ed8c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:22.159076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61371-0b11-4e81-9a7e-3d57c4ed8c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:22.159361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61371-0b11-4e81-9a7e-3d57c4ed8c22 admin admin] PATH_INFO: `/v3/projects/6feca6988aa74d0ca67f74e2b6a93796/users/d4aab2437e0a407aa21dd3062d0d72c5/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:22.164527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c61371-0b11-4e81-9a7e-3d57c4ed8c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6feca6988aa74d0ca67f74e2b6a93796, user_id=d4aab2437e0a407aa21dd3062d0d72c5, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:22.174098 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c61371-0b11-4e81-9a7e-3d57c4ed8c22 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:22.199950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 833/1654] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:22 2022] PUT /identity/v3/projects/6feca6988aa74d0ca67f74e2b6a93796/users/d4aab2437e0a407aa21dd3062d0d72c5/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:22.204473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd717c9-047b-4497-86d1-81bfc17261ef admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:22.280784 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:22.281443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:22.281842 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:22.282159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] PATH_INFO: `/v3/users/d89c4aa62b3f4247a2d76011f7b2d12c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:22.283427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89c4aa62b3f4247a2d76011f7b2d12c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:22.286051 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:22.382444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:22.549636 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89c4aa62b3f4247a2d76011f7b2d12c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:22.621737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89c4aa62b3f4247a2d76011f7b2d12c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:22.660921 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89c4aa62b3f4247a2d76011f7b2d12c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:22.662980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89c4aa62b3f4247a2d76011f7b2d12c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:22.663181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89c4aa62b3f4247a2d76011f7b2d12c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:22.704074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc560a08-2c03-40cf-bcdc-21ec576a78c5 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:22.705165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc560a08-2c03-40cf-bcdc-21ec576a78c5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:22.705644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc560a08-2c03-40cf-bcdc-21ec576a78c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:22.709092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 822/1655] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:37:22 2022] DELETE /identity/v3/users/d89c4aa62b3f4247a2d76011f7b2d12c => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:22.740815 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-dc560a08-2c03-40cf-bcdc-21ec576a78c5 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:22.743931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dc560a08-2c03-40cf-bcdc-21ec576a78c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:22.904402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 834/1656] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:22.999865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:23.001566 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:23.116959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd5ca-a44e-4d0f-8d45-a77e4c3f9446 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:23.127833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37422e73-3121-4c2f-abf4-05fa4349dc73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:23.129301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422e73-3121-4c2f-abf4-05fa4349dc73 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:23.129763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422e73-3121-4c2f-abf4-05fa4349dc73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:23.130224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422e73-3121-4c2f-abf4-05fa4349dc73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:23.173372 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37422e73-3121-4c2f-abf4-05fa4349dc73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:23.180936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37422e73-3121-4c2f-abf4-05fa4349dc73 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:23.235437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 823/1657] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:23.648896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc560a08-2c03-40cf-bcdc-21ec576a78c5 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:23.653902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dc560a08-2c03-40cf-bcdc-21ec576a78c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:23.677947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc560a08-2c03-40cf-bcdc-21ec576a78c5 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:23.700125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:23.700829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:23.701172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:23.701413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] PATH_INFO: `/v3/projects/6fcdd19a618d41f9b498017b0197f4fd` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:23.709180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fcdd19a618d41f9b498017b0197f4fd)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:23.722114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:23.912794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fcdd19a618d41f9b498017b0197f4fd', 'request_id': 'req-e413b1f7-1d70-443b-bee4-677de6083ac1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:23.928759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fcdd19a618d41f9b498017b0197f4fd', 'request_id': 'req-e413b1f7-1d70-443b-bee4-677de6083ac1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:24.045110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fcdd19a618d41f9b498017b0197f4fd', 'request_id': 'req-e413b1f7-1d70-443b-bee4-677de6083ac1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:24.070698 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] The token cache is being invalidate because project 6fcdd19a618d41f9b498017b0197f4fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:37:24.071495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:24.072457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e413b1f7-1d70-443b-bee4-677de6083ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:24.079757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 835/1658] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:37:23 2022] DELETE /identity/v3/projects/6fcdd19a618d41f9b498017b0197f4fd => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:24.082158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c86e5-c8aa-4bb2-96c4-76d29daa6629 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:24.085408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c86e5-c8aa-4bb2-96c4-76d29daa6629 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:24.085854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c86e5-c8aa-4bb2-96c4-76d29daa6629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:24.137351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0b7c86e5-c8aa-4bb2-96c4-76d29daa6629 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:24.138808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0b7c86e5-c8aa-4bb2-96c4-76d29daa6629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:24.325383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 824/1659] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:24.336947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bd0f6-747b-45ec-be8e-c62175bc76a3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:24.337438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bd0f6-747b-45ec-be8e-c62175bc76a3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:24.337782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bd0f6-747b-45ec-be8e-c62175bc76a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:24.380992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ff2bd0f6-747b-45ec-be8e-c62175bc76a3 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:24.382769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ff2bd0f6-747b-45ec-be8e-c62175bc76a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:24.404817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 836/1660] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:24.406159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c86e5-c8aa-4bb2-96c4-76d29daa6629 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:24.437281 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c86e5-c8aa-4bb2-96c4-76d29daa6629 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:24.449578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b208c1e-6915-47ee-a858-d43ef4923367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:24.450318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b208c1e-6915-47ee-a858-d43ef4923367 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:24.450631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b208c1e-6915-47ee-a858-d43ef4923367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:24.450908 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b208c1e-6915-47ee-a858-d43ef4923367 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:24.451625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b208c1e-6915-47ee-a858-d43ef4923367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:24.451976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b208c1e-6915-47ee-a858-d43ef4923367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:24.453643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b208c1e-6915-47ee-a858-d43ef4923367 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:24.459026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:24.505748 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bd0f6-747b-45ec-be8e-c62175bc76a3 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:24.507744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 825/1661] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:24.509131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ff2bd0f6-747b-45ec-be8e-c62175bc76a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:24.533557 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b208c1e-6915-47ee-a858-d43ef4923367 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:24.553811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bd0f6-747b-45ec-be8e-c62175bc76a3 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:24.569934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b208c1e-6915-47ee-a858-d43ef4923367 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:24.583485 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:24.584176 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:24.584459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:24.584737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] PATH_INFO: `/v3/users/c461b1dffb1a412b9fb69bb3b2c4a5bd` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:24.586293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c461b1dffb1a412b9fb69bb3b2c4a5bd)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:24.587947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:37:24.587947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Aug 07 22:37:24.587947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: Aug 07 22:37:24.589219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:24.600929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73537e11-a106-4399-8a4c-218621579328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:24.603317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73537e11-a106-4399-8a4c-218621579328 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:24.603317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73537e11-a106-4399-8a4c-218621579328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:24.603317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73537e11-a106-4399-8a4c-218621579328 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:24.605345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73537e11-a106-4399-8a4c-218621579328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:24.607512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73537e11-a106-4399-8a4c-218621579328 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:24.807755 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 826/1662] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:24 2022] POST /identity/v3/projects => generated 373 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:37:24.808797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73537e11-a106-4399-8a4c-218621579328 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:24.885882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:24.920680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c461b1dffb1a412b9fb69bb3b2c4a5bd'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:24.943894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73537e11-a106-4399-8a4c-218621579328 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:24.949134 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a802bd-d876-44f2-9676-46174dc85190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:24.949803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a802bd-d876-44f2-9676-46174dc85190 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:24.950081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a802bd-d876-44f2-9676-46174dc85190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:24.950360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a802bd-d876-44f2-9676-46174dc85190 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:24.950984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a802bd-d876-44f2-9676-46174dc85190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:24.953003 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a802bd-d876-44f2-9676-46174dc85190 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:24.997560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c461b1dffb1a412b9fb69bb3b2c4a5bd'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:25.033433 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 827/1663] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:24 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:25.038001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a802bd-d876-44f2-9676-46174dc85190 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:25.051505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c461b1dffb1a412b9fb69bb3b2c4a5bd'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:25.060710 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a802bd-d876-44f2-9676-46174dc85190 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:25.071556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7021e0-5f5f-442f-a27c-1c64b1357a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:25.073230 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7021e0-5f5f-442f-a27c-1c64b1357a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:25.073520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7021e0-5f5f-442f-a27c-1c64b1357a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:25.073792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7021e0-5f5f-442f-a27c-1c64b1357a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:25.075648 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7021e0-5f5f-442f-a27c-1c64b1357a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:25.081500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7021e0-5f5f-442f-a27c-1c64b1357a7c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:25.091401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c461b1dffb1a412b9fb69bb3b2c4a5bd'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:25.096522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-90513327-5a89-4845-a414-d16f3f869410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c461b1dffb1a412b9fb69bb3b2c4a5bd'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:25.109633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 828/1664] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:25.113977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 837/1665] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:37:24 2022] DELETE /identity/v3/users/c461b1dffb1a412b9fb69bb3b2c4a5bd => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:25.119129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7021e0-5f5f-442f-a27c-1c64b1357a7c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:25.120657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c95c91-3ff9-46bc-b07b-7a4f6dbdb15a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:25.120957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c95c91-3ff9-46bc-b07b-7a4f6dbdb15a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:25.121303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c95c91-3ff9-46bc-b07b-7a4f6dbdb15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:25.121389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ed7021e0-5f5f-442f-a27c-1c64b1357a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:25.137639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7021e0-5f5f-442f-a27c-1c64b1357a7c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:25.175672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f6c95c91-3ff9-46bc-b07b-7a4f6dbdb15a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:25.178222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f6c95c91-3ff9-46bc-b07b-7a4f6dbdb15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:25.274790 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b78a5-20c9-48e7-a1e2-0d9e7e3020ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:25.275696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b78a5-20c9-48e7-a1e2-0d9e7e3020ef admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:25.276110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b78a5-20c9-48e7-a1e2-0d9e7e3020ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:25.276494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b78a5-20c9-48e7-a1e2-0d9e7e3020ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:25.277227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b78a5-20c9-48e7-a1e2-0d9e7e3020ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:25.279930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b78a5-20c9-48e7-a1e2-0d9e7e3020ef admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:25.296081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 829/1666] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:25.297536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b78a5-20c9-48e7-a1e2-0d9e7e3020ef admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:25.314306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b78a5-20c9-48e7-a1e2-0d9e7e3020ef admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:25.320734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 838/1667] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:37:25.322415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7866f-13e1-420c-8553-75e6e602bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:25.329933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7866f-13e1-420c-8553-75e6e602bc4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:25.335746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7866f-13e1-420c-8553-75e6e602bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:25.335746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7866f-13e1-420c-8553-75e6e602bc4e admin admin] PATH_INFO: `/v3/projects/6f0c260ad9934d7ab8d7e8d54518a74e/users/eb1dc96663464c1daf3253d154dacebf/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:25.387330 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7866f-13e1-420c-8553-75e6e602bc4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f0c260ad9934d7ab8d7e8d54518a74e, user_id=eb1dc96663464c1daf3253d154dacebf, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:25.399441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7866f-13e1-420c-8553-75e6e602bc4e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:25.413782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c95c91-3ff9-46bc-b07b-7a4f6dbdb15a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:25.417311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f6c95c91-3ff9-46bc-b07b-7a4f6dbdb15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:25.516265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 830/1668] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:25 2022] PUT /identity/v3/projects/6f0c260ad9934d7ab8d7e8d54518a74e/users/eb1dc96663464c1daf3253d154dacebf/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:25.527908 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b23fad-3344-4a17-baba-9e95ace786d5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:25.528233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b23fad-3344-4a17-baba-9e95ace786d5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:25.528498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b23fad-3344-4a17-baba-9e95ace786d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:25.588842 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b9b23fad-3344-4a17-baba-9e95ace786d5 None None] MFA Rules not processed for user `eb1dc96663464c1daf3253d154dacebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:25.592367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c95c91-3ff9-46bc-b07b-7a4f6dbdb15a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:25.603036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b9b23fad-3344-4a17-baba-9e95ace786d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:25.651289 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:25.759318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61d840-c0e6-4fae-9850-a66f29cb50c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:25.762154 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61d840-c0e6-4fae-9850-a66f29cb50c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:25.762439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61d840-c0e6-4fae-9850-a66f29cb50c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:25.762713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61d840-c0e6-4fae-9850-a66f29cb50c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:25.813861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 831/1669] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:25.814910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b23fad-3344-4a17-baba-9e95ace786d5 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:25.819169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b9b23fad-3344-4a17-baba-9e95ace786d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:25.832783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:25.909170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61d840-c0e6-4fae-9850-a66f29cb50c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:25.912036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61d840-c0e6-4fae-9850-a66f29cb50c7 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:25.943669 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 839/1670] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:25.945308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61d840-c0e6-4fae-9850-a66f29cb50c7 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:25.947531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-aa61d840-c0e6-4fae-9850-a66f29cb50c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:25.956280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b23fad-3344-4a17-baba-9e95ace786d5 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:25.962830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:25.963734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:25.964125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:25.964504 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] PATH_INFO: `/v3/projects/b89549eee7fd4f2ba1657576213f254a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:25.980501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b89549eee7fd4f2ba1657576213f254a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:25.981215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 22:37:25.981215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Aug 07 22:37:25.981215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: Aug 07 22:37:25.982602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:26.087835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61d840-c0e6-4fae-9850-a66f29cb50c7 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:26.102456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d1cd0-0f67-4bb4-8c83-76af06773321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:26.104399 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1cd0-0f67-4bb4-8c83-76af06773321 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:26.105080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1cd0-0f67-4bb4-8c83-76af06773321 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:26.105627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1cd0-0f67-4bb4-8c83-76af06773321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:26.124798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d1cd0-0f67-4bb4-8c83-76af06773321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:26.126737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d1cd0-0f67-4bb4-8c83-76af06773321 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:26.143807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 840/1671] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:26.146603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b89549eee7fd4f2ba1657576213f254a', 'request_id': 'req-2b360af5-499d-48bc-89b6-35929a45e118'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:26.182738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b89549eee7fd4f2ba1657576213f254a', 'request_id': 'req-2b360af5-499d-48bc-89b6-35929a45e118'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:26.296966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b89549eee7fd4f2ba1657576213f254a', 'request_id': 'req-2b360af5-499d-48bc-89b6-35929a45e118'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:26.313175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] The token cache is being invalidate because project b89549eee7fd4f2ba1657576213f254a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:37:26.313479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:26.314744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2b360af5-499d-48bc-89b6-35929a45e118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:26.330409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 832/1672] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:37:25 2022] DELETE /identity/v3/projects/b89549eee7fd4f2ba1657576213f254a => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:26.334533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d1cd0-0f67-4bb4-8c83-76af06773321 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:26.354002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d1cd0-0f67-4bb4-8c83-76af06773321 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:26.529225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae9009-1543-4d38-aec7-0d90346b4049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:26.530221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae9009-1543-4d38-aec7-0d90346b4049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:26.530625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae9009-1543-4d38-aec7-0d90346b4049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:26.530959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae9009-1543-4d38-aec7-0d90346b4049 admin admin] PATH_INFO: `/v3/users/c45bb9bee57a4fe9a99643dca9565f1e` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:26.533168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae9009-1543-4d38-aec7-0d90346b4049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c45bb9bee57a4fe9a99643dca9565f1e)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:26.536304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae9009-1543-4d38-aec7-0d90346b4049 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:26.597188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4df97f-b435-45a5-bacb-6aa950eae94f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:26.598624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4df97f-b435-45a5-bacb-6aa950eae94f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:26.599061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4df97f-b435-45a5-bacb-6aa950eae94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:26.645715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4c4df97f-b435-45a5-bacb-6aa950eae94f None None] MFA Rules not processed for user `d4aab2437e0a407aa21dd3062d0d72c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:26.647595 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4c4df97f-b435-45a5-bacb-6aa950eae94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:26.816985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 833/1673] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:26 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:26.844654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f7ae9009-1543-4d38-aec7-0d90346b4049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c45bb9bee57a4fe9a99643dca9565f1e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:26.925554 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:26.953903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f7ae9009-1543-4d38-aec7-0d90346b4049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c45bb9bee57a4fe9a99643dca9565f1e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:26.980998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f7ae9009-1543-4d38-aec7-0d90346b4049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c45bb9bee57a4fe9a99643dca9565f1e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:26.990345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f7ae9009-1543-4d38-aec7-0d90346b4049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c45bb9bee57a4fe9a99643dca9565f1e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:26.991095 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f7ae9009-1543-4d38-aec7-0d90346b4049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c45bb9bee57a4fe9a99643dca9565f1e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:27.000367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 841/1674] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:37:26 2022] DELETE /identity/v3/users/c45bb9bee57a4fe9a99643dca9565f1e => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:27.324714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4df97f-b435-45a5-bacb-6aa950eae94f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:27.326351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4c4df97f-b435-45a5-bacb-6aa950eae94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:27.352363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4df97f-b435-45a5-bacb-6aa950eae94f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:27.507325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:27.507989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:27.508508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:27.508794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] PATH_INFO: `/v3/projects/bf8ae1dcb7bb4d58b383f8e992528f62` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:27.511637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf8ae1dcb7bb4d58b383f8e992528f62)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:27.515510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:27.686021 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf8ae1dcb7bb4d58b383f8e992528f62', 'request_id': 'req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:27.699343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf8ae1dcb7bb4d58b383f8e992528f62', 'request_id': 'req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:27.810677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf8ae1dcb7bb4d58b383f8e992528f62', 'request_id': 'req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:27.829495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] The token cache is being invalidate because project bf8ae1dcb7bb4d58b383f8e992528f62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:37:27.830116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:27.832501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-df40766c-42bf-4ba7-9879-3d3a95d5a9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:27.835921 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 834/1675] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:37:27 2022] DELETE /identity/v3/projects/bf8ae1dcb7bb4d58b383f8e992528f62 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:27.841184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d5a7c-e2ba-4f5e-b7f0-c8eafa432c1b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:27.841552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d5a7c-e2ba-4f5e-b7f0-c8eafa432c1b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:27.841932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d5a7c-e2ba-4f5e-b7f0-c8eafa432c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:27.882452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-184d5a7c-e2ba-4f5e-b7f0-c8eafa432c1b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:27.884163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-184d5a7c-e2ba-4f5e-b7f0-c8eafa432c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:27.997079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:28.049916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 842/1676] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:28.052441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105e1f9-df23-4748-9d5f-c16787d10869 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:28.052833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105e1f9-df23-4748-9d5f-c16787d10869 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:28.053229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105e1f9-df23-4748-9d5f-c16787d10869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:28.099176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b105e1f9-df23-4748-9d5f-c16787d10869 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:28.105470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b105e1f9-df23-4748-9d5f-c16787d10869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:28.131491 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 835/1677] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:28.136635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d5a7c-e2ba-4f5e-b7f0-c8eafa432c1b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:28.153905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d5a7c-e2ba-4f5e-b7f0-c8eafa432c1b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:28.166092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2362a51-2efe-4faf-9808-4821774b4e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:28.167084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2362a51-2efe-4faf-9808-4821774b4e96 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:28.167528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2362a51-2efe-4faf-9808-4821774b4e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:28.169360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2362a51-2efe-4faf-9808-4821774b4e96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:28.170443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2362a51-2efe-4faf-9808-4821774b4e96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:28.170975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2362a51-2efe-4faf-9808-4821774b4e96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:28.174918 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2362a51-2efe-4faf-9808-4821774b4e96 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:28.209201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 843/1678] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:28.212716 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105e1f9-df23-4748-9d5f-c16787d10869 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:28.242190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105e1f9-df23-4748-9d5f-c16787d10869 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:28.266866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b44a2-bfbb-414c-ac35-d51b3ec26ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:28.267839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b44a2-bfbb-414c-ac35-d51b3ec26ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:28.268221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b44a2-bfbb-414c-ac35-d51b3ec26ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:28.270696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b44a2-bfbb-414c-ac35-d51b3ec26ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:28.271397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b44a2-bfbb-414c-ac35-d51b3ec26ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:28.277826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b44a2-bfbb-414c-ac35-d51b3ec26ae4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:28.489634 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 836/1679] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:28 2022] POST /identity/v3/projects => generated 367 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:28.494183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2362a51-2efe-4faf-9808-4821774b4e96 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:28.518883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2362a51-2efe-4faf-9808-4821774b4e96 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:28.657923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b6a32-4bf0-4fb1-95f7-66aafb930c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:28.658739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b6a32-4bf0-4fb1-95f7-66aafb930c36 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:28.659117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b6a32-4bf0-4fb1-95f7-66aafb930c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:28.659491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b6a32-4bf0-4fb1-95f7-66aafb930c36 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:28.660268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b6a32-4bf0-4fb1-95f7-66aafb930c36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:28.662739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b6a32-4bf0-4fb1-95f7-66aafb930c36 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:28.785715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b44a2-bfbb-414c-ac35-d51b3ec26ae4 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:28.786160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 844/1680] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:28 2022] POST /identity/v3/users => generated 284 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:28.796869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b44a2-bfbb-414c-ac35-d51b3ec26ae4 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:28.802833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a81e03-2645-4fc7-84f6-1761801db3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:28.807658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a81e03-2645-4fc7-84f6-1761801db3af admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:28.807658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a81e03-2645-4fc7-84f6-1761801db3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:28.807658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a81e03-2645-4fc7-84f6-1761801db3af admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:28.807658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a81e03-2645-4fc7-84f6-1761801db3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:28.807658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a81e03-2645-4fc7-84f6-1761801db3af admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:28.821782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 837/1681] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:28.825349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b6a32-4bf0-4fb1-95f7-66aafb930c36 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:28.838756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b6a32-4bf0-4fb1-95f7-66aafb930c36 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:28.849438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02f7ac-6ba9-419d-9fce-970255a098fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:28.850353 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02f7ac-6ba9-419d-9fce-970255a098fd admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:28.851539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02f7ac-6ba9-419d-9fce-970255a098fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:28.852129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02f7ac-6ba9-419d-9fce-970255a098fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:28.855113 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02f7ac-6ba9-419d-9fce-970255a098fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:28.859311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02f7ac-6ba9-419d-9fce-970255a098fd admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:28.887380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 845/1682] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:28.893151 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a81e03-2645-4fc7-84f6-1761801db3af admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:28.904868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a81e03-2645-4fc7-84f6-1761801db3af admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:28.910960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a8070-7fd8-455b-be92-e057eacdbea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:28.912381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a8070-7fd8-455b-be92-e057eacdbea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:28.912813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a8070-7fd8-455b-be92-e057eacdbea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:28.913257 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a8070-7fd8-455b-be92-e057eacdbea7 admin admin] PATH_INFO: `/v3/projects/09e9e6e1075c4da8b09ef1358c7a335e/users/2270e3b5a8b445d98c29ecb71a3f1927/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:28.944548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a8070-7fd8-455b-be92-e057eacdbea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e9e6e1075c4da8b09ef1358c7a335e, user_id=2270e3b5a8b445d98c29ecb71a3f1927, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:28.949081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a8070-7fd8-455b-be92-e057eacdbea7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:28.954123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:29.052844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 838/1683] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:28 2022] PUT /identity/v3/projects/09e9e6e1075c4da8b09ef1358c7a335e/users/2270e3b5a8b445d98c29ecb71a3f1927/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:29.054491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d0ef-370d-474a-8aab-5394579df00c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:29.054961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d0ef-370d-474a-8aab-5394579df00c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:29.056366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d0ef-370d-474a-8aab-5394579df00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:29.123588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b224d0ef-370d-474a-8aab-5394579df00c None None] MFA Rules not processed for user `2270e3b5a8b445d98c29ecb71a3f1927`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:29.124897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b224d0ef-370d-474a-8aab-5394579df00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:29.287568 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 846/1684] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:37:29.299204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a8070-7fd8-455b-be92-e057eacdbea7 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:29.301564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3b6a8070-7fd8-455b-be92-e057eacdbea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:29.319275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:29.396206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a8070-7fd8-455b-be92-e057eacdbea7 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:29.400423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18c9ef-7767-4c28-921b-9d3294165610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:29.401286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c9ef-7767-4c28-921b-9d3294165610 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:29.401711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c9ef-7767-4c28-921b-9d3294165610 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:29.402131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c9ef-7767-4c28-921b-9d3294165610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:29.411918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18c9ef-7767-4c28-921b-9d3294165610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:29.413640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18c9ef-7767-4c28-921b-9d3294165610 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:29.424859 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 839/1685] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:29.543736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f8dbb-3c06-4cc0-9f3d-6bf5bb29ebba None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:29.544191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f8dbb-3c06-4cc0-9f3d-6bf5bb29ebba None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:29.544773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f8dbb-3c06-4cc0-9f3d-6bf5bb29ebba None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:29.549364 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 847/1686] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:37:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:37:29.632332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f0649-80cb-4f68-9e93-e09de17e6ba4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:29.633111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f8dbb-3c06-4cc0-9f3d-6bf5bb29ebba None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:29.636780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f0649-80cb-4f68-9e93-e09de17e6ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:29.638149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-131f8dbb-3c06-4cc0-9f3d-6bf5bb29ebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:29.638431 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f0649-80cb-4f68-9e93-e09de17e6ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:29.694832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a79f0649-80cb-4f68-9e93-e09de17e6ba4 None None] MFA Rules not processed for user `db1d7e39e8c14f5db40efa743fc3b652`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:29.696792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a79f0649-80cb-4f68-9e93-e09de17e6ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:29.785884 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f8dbb-3c06-4cc0-9f3d-6bf5bb29ebba None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:29.801305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:29.802269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:29.802558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:29.802834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] PATH_INFO: `/v3/projects/e6a6be36471846218b7cdd351274d071` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:29.804418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a6be36471846218b7cdd351274d071)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:29.808691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:29.866928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 840/1687] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:37:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:29.880333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8584d8-20e8-483b-be55-f0ae1445cdd3 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:29.882968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8584d8-20e8-483b-be55-f0ae1445cdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:29.887910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8584d8-20e8-483b-be55-f0ae1445cdd3 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:29.904293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 841/1688] 10.222.0.30 () {54 vars in 961 bytes} [Sun Aug 7 22:37:29 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:37:29.911343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd212e6-da03-49e8-b23a-9f8d2c251b1f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:29.913281 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd212e6-da03-49e8-b23a-9f8d2c251b1f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:29.915251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd212e6-da03-49e8-b23a-9f8d2c251b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:29.965099 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2cd212e6-da03-49e8-b23a-9f8d2c251b1f None None] MFA Rules not processed for user `db1d7e39e8c14f5db40efa743fc3b652`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:29.967044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2cd212e6-da03-49e8-b23a-9f8d2c251b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:30.011117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a6be36471846218b7cdd351274d071', 'request_id': 'req-ca3a019c-a809-4e4b-98f2-9c24728fab75'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:30.023285 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a6be36471846218b7cdd351274d071', 'request_id': 'req-ca3a019c-a809-4e4b-98f2-9c24728fab75'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:30.092058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a6be36471846218b7cdd351274d071', 'request_id': 'req-ca3a019c-a809-4e4b-98f2-9c24728fab75'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:30.113458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] The token cache is being invalidate because project e6a6be36471846218b7cdd351274d071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:37:30.113928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:30.115341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ca3a019c-a809-4e4b-98f2-9c24728fab75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:30.125987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 848/1689] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:37:29 2022] DELETE /identity/v3/projects/e6a6be36471846218b7cdd351274d071 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:30.129998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127e2ce-e391-499e-a0d6-557c82ab307d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:30.130647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127e2ce-e391-499e-a0d6-557c82ab307d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:30.131119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127e2ce-e391-499e-a0d6-557c82ab307d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:30.157867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 842/1690] 10.222.0.30 () {58 vars in 1058 bytes} [Sun Aug 7 22:37:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:30.162295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd212e6-da03-49e8-b23a-9f8d2c251b1f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:30.172911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f127e2ce-e391-499e-a0d6-557c82ab307d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:30.174436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f127e2ce-e391-499e-a0d6-557c82ab307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:30.175719 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd212e6-da03-49e8-b23a-9f8d2c251b1f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:30.183622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f15a40-b8e1-401c-a16e-ba64e4e5d6b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:30.184695 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f15a40-b8e1-401c-a16e-ba64e4e5d6b2 service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:30.185229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f15a40-b8e1-401c-a16e-ba64e4e5d6b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:30.185714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f15a40-b8e1-401c-a16e-ba64e4e5d6b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:30.192072 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-73f15a40-b8e1-401c-a16e-ba64e4e5d6b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:30.207765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f15a40-b8e1-401c-a16e-ba64e4e5d6b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:30.210188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f15a40-b8e1-401c-a16e-ba64e4e5d6b2 service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:30.256035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 843/1691] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:30.313048 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 849/1692] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:37:30.323651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd296f-6f24-4edb-89a2-a2204b785fe0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:30.324315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd296f-6f24-4edb-89a2-a2204b785fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:30.324809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd296f-6f24-4edb-89a2-a2204b785fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:30.368458 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-57dd296f-6f24-4edb-89a2-a2204b785fe0 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:30.370067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-57dd296f-6f24-4edb-89a2-a2204b785fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:30.396559 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 844/1693] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:30.397406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127e2ce-e391-499e-a0d6-557c82ab307d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:30.410343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127e2ce-e391-499e-a0d6-557c82ab307d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:30.425652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c711-6b02-447b-90b6-03ae08f6d85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:30.426305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c711-6b02-447b-90b6-03ae08f6d85a admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:30.426584 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c711-6b02-447b-90b6-03ae08f6d85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:30.427201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c711-6b02-447b-90b6-03ae08f6d85a admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:30.427948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c711-6b02-447b-90b6-03ae08f6d85a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:30.428597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c711-6b02-447b-90b6-03ae08f6d85a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:30.433214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c711-6b02-447b-90b6-03ae08f6d85a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:30.477632 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 850/1694] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:30.478944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd296f-6f24-4edb-89a2-a2204b785fe0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:30.501802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd296f-6f24-4edb-89a2-a2204b785fe0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:30.525348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b908af4-a7c3-411b-873a-b66ef0407e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:30.526342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b908af4-a7c3-411b-873a-b66ef0407e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:30.526997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b908af4-a7c3-411b-873a-b66ef0407e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:30.527468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b908af4-a7c3-411b-873a-b66ef0407e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:30.528445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b908af4-a7c3-411b-873a-b66ef0407e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:30.530697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b908af4-a7c3-411b-873a-b66ef0407e45 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:30.551874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:30.677228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 845/1695] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:30 2022] POST /identity/v3/projects => generated 357 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:37:30.681213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c711-6b02-447b-90b6-03ae08f6d85a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:30.694449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c711-6b02-447b-90b6-03ae08f6d85a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:30.817887 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e3cea-9ea7-4284-a25c-11286fb8db72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:30.818767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e3cea-9ea7-4284-a25c-11286fb8db72 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:30.819183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e3cea-9ea7-4284-a25c-11286fb8db72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:30.819648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e3cea-9ea7-4284-a25c-11286fb8db72 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:30.820547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e3cea-9ea7-4284-a25c-11286fb8db72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:30.824206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e3cea-9ea7-4284-a25c-11286fb8db72 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:31.009600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17d3cd-6933-4d9f-8b72-4e2902a16379 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:31.010906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17d3cd-6933-4d9f-8b72-4e2902a16379 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:31.014193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17d3cd-6933-4d9f-8b72-4e2902a16379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:31.027445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 851/1696] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:30 2022] POST /identity/v3/users => generated 279 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:37:31.032230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e3cea-9ea7-4284-a25c-11286fb8db72 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:31.035169 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:31.074606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e3cea-9ea7-4284-a25c-11286fb8db72 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:31.083423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483ee1d-9e27-4de9-8850-8386566cbb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:31.086115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483ee1d-9e27-4de9-8850-8386566cbb05 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:31.088239 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483ee1d-9e27-4de9-8850-8386566cbb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:31.088688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483ee1d-9e27-4de9-8850-8386566cbb05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:31.090857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483ee1d-9e27-4de9-8850-8386566cbb05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:31.094016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6e17d3cd-6933-4d9f-8b72-4e2902a16379 None None] MFA Rules not processed for user `dfd02222f0b14a588b51a792b02ca852`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:31.095500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6e17d3cd-6933-4d9f-8b72-4e2902a16379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:31.095779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483ee1d-9e27-4de9-8850-8386566cbb05 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:31.157552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 852/1697] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:31.162858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483ee1d-9e27-4de9-8850-8386566cbb05 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:31.205461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483ee1d-9e27-4de9-8850-8386566cbb05 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:31.228091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80b6fd-9d91-4f5e-8d23-a7497da6e5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:31.229913 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80b6fd-9d91-4f5e-8d23-a7497da6e5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:31.230952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80b6fd-9d91-4f5e-8d23-a7497da6e5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:31.235512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80b6fd-9d91-4f5e-8d23-a7497da6e5e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:31.235512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80b6fd-9d91-4f5e-8d23-a7497da6e5e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:31.235512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80b6fd-9d91-4f5e-8d23-a7497da6e5e3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:31.267473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 853/1698] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:31.273255 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80b6fd-9d91-4f5e-8d23-a7497da6e5e3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:31.298108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80b6fd-9d91-4f5e-8d23-a7497da6e5e3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:31.312619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc832ad2-0115-466a-bcae-1347b53e8d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:31.313304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc832ad2-0115-466a-bcae-1347b53e8d73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:31.313614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc832ad2-0115-466a-bcae-1347b53e8d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:31.313860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc832ad2-0115-466a-bcae-1347b53e8d73 admin admin] PATH_INFO: `/v3/projects/9bd51ffb38e0416da76cbf1771b2be9f/users/52d15ab0003c4b9e86c614bf18b7700e/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:31.348424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 846/1699] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:31.361873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ea656-327e-4169-ab01-4a4dac89455e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:31.363316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ea656-327e-4169-ab01-4a4dac89455e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:31.364564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ea656-327e-4169-ab01-4a4dac89455e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:31.372897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc832ad2-0115-466a-bcae-1347b53e8d73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd51ffb38e0416da76cbf1771b2be9f, user_id=52d15ab0003c4b9e86c614bf18b7700e, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:31.381679 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc832ad2-0115-466a-bcae-1347b53e8d73 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:31.421294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-077ea656-327e-4169-ab01-4a4dac89455e None None] MFA Rules not processed for user `dfd02222f0b14a588b51a792b02ca852`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:31.424513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-077ea656-327e-4169-ab01-4a4dac89455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:31.475151 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 847/1700] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:31.483794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 854/1701] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:31 2022] PUT /identity/v3/projects/9bd51ffb38e0416da76cbf1771b2be9f/users/52d15ab0003c4b9e86c614bf18b7700e/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:31.485884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ea656-327e-4169-ab01-4a4dac89455e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:31.495706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdde8e5-67dd-4dd1-b429-b062830a3d08 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:31.496317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdde8e5-67dd-4dd1-b429-b062830a3d08 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:31.496857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdde8e5-67dd-4dd1-b429-b062830a3d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:31.501880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-077ea656-327e-4169-ab01-4a4dac89455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:31.567212 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-dbdde8e5-67dd-4dd1-b429-b062830a3d08 None None] MFA Rules not processed for user `52d15ab0003c4b9e86c614bf18b7700e`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:31.569043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dbdde8e5-67dd-4dd1-b429-b062830a3d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:31.748612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ea656-327e-4169-ab01-4a4dac89455e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:31.795950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 855/1702] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:31.800452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdde8e5-67dd-4dd1-b429-b062830a3d08 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:31.959017 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fced642-02d6-4215-bf0d-a9bba86f0cf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:31.959625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fced642-02d6-4215-bf0d-a9bba86f0cf2 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:31.959908 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fced642-02d6-4215-bf0d-a9bba86f0cf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:31.960306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fced642-02d6-4215-bf0d-a9bba86f0cf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:31.974717 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:32.034484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdde8e5-67dd-4dd1-b429-b062830a3d08 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:32.044837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa9c68-aa3b-4e26-84e9-4bb2cf288cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:32.045487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa9c68-aa3b-4e26-84e9-4bb2cf288cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:32.045770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa9c68-aa3b-4e26-84e9-4bb2cf288cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:32.046050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa9c68-aa3b-4e26-84e9-4bb2cf288cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:32.046800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa9c68-aa3b-4e26-84e9-4bb2cf288cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:32.059839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa9c68-aa3b-4e26-84e9-4bb2cf288cb2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:32.136359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fced642-02d6-4215-bf0d-a9bba86f0cf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:32.136359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fced642-02d6-4215-bf0d-a9bba86f0cf2 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:32.174136 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 848/1703] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:32.177366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fced642-02d6-4215-bf0d-a9bba86f0cf2 service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:32.178801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1fced642-02d6-4215-bf0d-a9bba86f0cf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:32.208088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 856/1704] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:31 2022] POST /identity/v3/projects => generated 355 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:32.209188 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa9c68-aa3b-4e26-84e9-4bb2cf288cb2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:32.222073 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fced642-02d6-4215-bf0d-a9bba86f0cf2 service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:32.385673 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa9c68-aa3b-4e26-84e9-4bb2cf288cb2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:32.393584 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec11b8b-eab5-4abe-a52c-f7da3496586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:32.394488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec11b8b-eab5-4abe-a52c-f7da3496586e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:32.394963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec11b8b-eab5-4abe-a52c-f7da3496586e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:32.395413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec11b8b-eab5-4abe-a52c-f7da3496586e admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:32.396237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec11b8b-eab5-4abe-a52c-f7da3496586e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:32.399881 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec11b8b-eab5-4abe-a52c-f7da3496586e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:32.435462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:32.439206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:32.439206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:32.439206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:32.439206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] PATH_INFO: `/v3/users/b79ec2edd94846ab973e8ece6b78aa89` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:32.441234 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b79ec2edd94846ab973e8ece6b78aa89)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:32.443686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:32.488247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 857/1705] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:32 2022] POST /identity/v3/users => generated 284 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:32.493061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec11b8b-eab5-4abe-a52c-f7da3496586e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:32.523480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec11b8b-eab5-4abe-a52c-f7da3496586e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:32.535355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062275a-a2bb-4ff9-8299-caca5ca37a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:32.535355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062275a-a2bb-4ff9-8299-caca5ca37a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:32.535355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062275a-a2bb-4ff9-8299-caca5ca37a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:32.535355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062275a-a2bb-4ff9-8299-caca5ca37a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:32.535644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062275a-a2bb-4ff9-8299-caca5ca37a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:32.537807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062275a-a2bb-4ff9-8299-caca5ca37a8c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:32.552088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 858/1706] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:32.564447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062275a-a2bb-4ff9-8299-caca5ca37a8c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:32.581000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062275a-a2bb-4ff9-8299-caca5ca37a8c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:32.588368 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734f7da-b601-4ba0-83c4-0b4459439563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:32.589179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734f7da-b601-4ba0-83c4-0b4459439563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:32.589524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734f7da-b601-4ba0-83c4-0b4459439563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:32.589876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734f7da-b601-4ba0-83c4-0b4459439563 admin admin] PATH_INFO: `/v3/projects/7d3bb6a60b4541f5829e93d5161c9a1a/users/6541b13be2e144859906f7fb799c8486/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:32.622573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734f7da-b601-4ba0-83c4-0b4459439563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d3bb6a60b4541f5829e93d5161c9a1a, user_id=6541b13be2e144859906f7fb799c8486, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:32.628268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734f7da-b601-4ba0-83c4-0b4459439563 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:32.732185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 859/1707] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:32 2022] PUT /identity/v3/projects/7d3bb6a60b4541f5829e93d5161c9a1a/users/6541b13be2e144859906f7fb799c8486/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:32.733803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e500c-d2e7-4112-9753-d8c1dd9c4ded None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:32.735116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e500c-d2e7-4112-9753-d8c1dd9c4ded None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:32.735508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e500c-d2e7-4112-9753-d8c1dd9c4ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:32.749911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b79ec2edd94846ab973e8ece6b78aa89'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:32.801758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-565e500c-d2e7-4112-9753-d8c1dd9c4ded None None] MFA Rules not processed for user `966045fbe69f43b4b1a825da3ad571b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:32.803555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-565e500c-d2e7-4112-9753-d8c1dd9c4ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:32.869885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b79ec2edd94846ab973e8ece6b78aa89'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:32.904209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b79ec2edd94846ab973e8ece6b78aa89'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:32.905060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b79ec2edd94846ab973e8ece6b78aa89'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:32.908476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b79ec2edd94846ab973e8ece6b78aa89'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:32.934735 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 849/1708] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:37:32 2022] DELETE /identity/v3/users/b79ec2edd94846ab973e8ece6b78aa89 => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:32.935889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:32.940849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:33.033397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 860/1709] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:33.040724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cb623-cffd-4ea6-b4ca-aa3cf7776e55 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:33.041019 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cb623-cffd-4ea6-b4ca-aa3cf7776e55 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:33.041287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cb623-cffd-4ea6-b4ca-aa3cf7776e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:33.066125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-741cb623-cffd-4ea6-b4ca-aa3cf7776e55 None None] MFA Rules not processed for user `966045fbe69f43b4b1a825da3ad571b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:33.067742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-741cb623-cffd-4ea6-b4ca-aa3cf7776e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:33.096499 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:33.161745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 861/1710] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:37:33.170755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cb623-cffd-4ea6-b4ca-aa3cf7776e55 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:33.172328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-741cb623-cffd-4ea6-b4ca-aa3cf7776e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:33.204387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389c1d5-2623-4cbe-82c8-2dca44d246dc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:33.218461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c508c11-c961-4eb8-aebc-fd09c93e9acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:33.218639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c508c11-c961-4eb8-aebc-fd09c93e9acc admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:33.218856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c508c11-c961-4eb8-aebc-fd09c93e9acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:33.219144 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c508c11-c961-4eb8-aebc-fd09c93e9acc admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:33.219814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c508c11-c961-4eb8-aebc-fd09c93e9acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:33.221485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c508c11-c961-4eb8-aebc-fd09c93e9acc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:33.287018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 850/1711] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:33.287629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c508c11-c961-4eb8-aebc-fd09c93e9acc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:33.292474 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cb623-cffd-4ea6-b4ca-aa3cf7776e55 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:33.307557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac069eb-1d44-429f-a977-e743ebde8d8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:33.311141 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac069eb-1d44-429f-a977-e743ebde8d8d service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:33.311380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac069eb-1d44-429f-a977-e743ebde8d8d service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:33.311796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac069eb-1d44-429f-a977-e743ebde8d8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:33.329756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c508c11-c961-4eb8-aebc-fd09c93e9acc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:33.338214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac069eb-1d44-429f-a977-e743ebde8d8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:33.339818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac069eb-1d44-429f-a977-e743ebde8d8d service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:33.347718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe375d-9a09-423d-969f-d169b0a67622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:33.348435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe375d-9a09-423d-969f-d169b0a67622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:33.348792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe375d-9a09-423d-969f-d169b0a67622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:33.354035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe375d-9a09-423d-969f-d169b0a67622 admin admin] PATH_INFO: `/v3/projects/7d3bb6a60b4541f5829e93d5161c9a1a/users/6541b13be2e144859906f7fb799c8486/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:33.356949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe375d-9a09-423d-969f-d169b0a67622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d3bb6a60b4541f5829e93d5161c9a1a, user_id=6541b13be2e144859906f7fb799c8486, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:33.361894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe375d-9a09-423d-969f-d169b0a67622 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:33.371742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 862/1712] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:33.390577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 851/1713] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:33 2022] PUT /identity/v3/projects/7d3bb6a60b4541f5829e93d5161c9a1a/users/6541b13be2e144859906f7fb799c8486/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:33.401777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908145a-342f-4476-979f-1346ed7194c6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:33.402209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908145a-342f-4476-979f-1346ed7194c6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:33.402605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908145a-342f-4476-979f-1346ed7194c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:33.453139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55640fd-fb3a-4239-88e4-436108329e81 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:33.453468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55640fd-fb3a-4239-88e4-436108329e81 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:33.453749 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55640fd-fb3a-4239-88e4-436108329e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:33.462974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:33.505798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e908145a-342f-4476-979f-1346ed7194c6 None None] MFA Rules not processed for user `6541b13be2e144859906f7fb799c8486`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:33.506386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e908145a-342f-4476-979f-1346ed7194c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:33.551966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d55640fd-fb3a-4239-88e4-436108329e81 None None] MFA Rules not processed for user `05ef5ec6afb84c4abc0cb4862f041dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:33.553254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d55640fd-fb3a-4239-88e4-436108329e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:33.730765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 863/1714] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:37:33.731980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4bb13-e8e5-4630-9754-d1a6665ed766 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:33.739059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4bb13-e8e5-4630-9754-d1a6665ed766 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:33.739059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4bb13-e8e5-4630-9754-d1a6665ed766 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:33.747577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 864/1715] 10.222.0.30 () {54 vars in 912 bytes} [Sun Aug 7 22:37:33 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:37:33.756348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c9202-7695-4da2-a285-9695f83c561a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:33.756518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c9202-7695-4da2-a285-9695f83c561a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:33.756754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c9202-7695-4da2-a285-9695f83c561a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:33.770625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 852/1716] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 07 22:37:33.772798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55640fd-fb3a-4239-88e4-436108329e81 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:33.813647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d55640fd-fb3a-4239-88e4-436108329e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:33.864354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-fc2c9202-7695-4da2-a285-9695f83c561a None None] MFA Rules not processed for user `baffd4f6a8ad4a6e91d8d9c61cc1dce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:33.872788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fc2c9202-7695-4da2-a285-9695f83c561a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:33.916355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55640fd-fb3a-4239-88e4-436108329e81 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:33.921307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa42e03-db3f-486c-a54d-7ec675fe1dad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:33.923587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa42e03-db3f-486c-a54d-7ec675fe1dad service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:33.924071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa42e03-db3f-486c-a54d-7ec675fe1dad service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:33.924510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa42e03-db3f-486c-a54d-7ec675fe1dad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:33.937834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa42e03-db3f-486c-a54d-7ec675fe1dad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:33.940317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa42e03-db3f-486c-a54d-7ec675fe1dad service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:33.970706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 853/1717] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:33.972989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa42e03-db3f-486c-a54d-7ec675fe1dad service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:33.978655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 865/1718] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:37:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:33.985945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cfa42e03-db3f-486c-a54d-7ec675fe1dad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:33.995458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c9202-7695-4da2-a285-9695f83c561a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:33.997131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fc2c9202-7695-4da2-a285-9695f83c561a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:34.013498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c9202-7695-4da2-a285-9695f83c561a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:34.029559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985a8d-588b-4472-b270-ff5a350131f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:34.030407 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985a8d-588b-4472-b270-ff5a350131f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:34.030881 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985a8d-588b-4472-b270-ff5a350131f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:34.031251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985a8d-588b-4472-b270-ff5a350131f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:34.042304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9985a8d-588b-4472-b270-ff5a350131f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:34.044843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9985a8d-588b-4472-b270-ff5a350131f4 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:34.060825 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 866/1719] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:34.122999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa42e03-db3f-486c-a54d-7ec675fe1dad service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:34.132920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:34.133646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:34.134079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:34.134358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] PATH_INFO: `/v3/projects/c1c8b88e7f944005a03d220e4e70c9cb` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:34.136088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c8b88e7f944005a03d220e4e70c9cb)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:34.138078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:34.166156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8528263-c0dc-4637-82cf-5f275293cfbd None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:34.166761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8528263-c0dc-4637-82cf-5f275293cfbd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:34.167203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8528263-c0dc-4637-82cf-5f275293cfbd None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:34.168754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 867/1720] 10.222.0.30 () {54 vars in 912 bytes} [Sun Aug 7 22:37:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:37:34.176088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed14f87-c0fc-4ba5-9c12-0e508a9c53d1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:34.176532 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed14f87-c0fc-4ba5-9c12-0e508a9c53d1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:34.177573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed14f87-c0fc-4ba5-9c12-0e508a9c53d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:34.226360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-fed14f87-c0fc-4ba5-9c12-0e508a9c53d1 None None] MFA Rules not processed for user `baffd4f6a8ad4a6e91d8d9c61cc1dce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:34.229485 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fed14f87-c0fc-4ba5-9c12-0e508a9c53d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:34.254852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 868/1721] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:37:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:37:34.279980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed14f87-c0fc-4ba5-9c12-0e508a9c53d1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:34.293414 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed14f87-c0fc-4ba5-9c12-0e508a9c53d1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:34.295761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c8b88e7f944005a03d220e4e70c9cb', 'request_id': 'req-44a80adc-6ed2-46cb-b7f0-e0554ea75932'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:34.304865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c8b88e7f944005a03d220e4e70c9cb', 'request_id': 'req-44a80adc-6ed2-46cb-b7f0-e0554ea75932'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:34.364986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf255104-4618-4142-9cd5-6fc46ea4dba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:34.367150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf255104-4618-4142-9cd5-6fc46ea4dba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:34.367633 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf255104-4618-4142-9cd5-6fc46ea4dba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:34.369043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf255104-4618-4142-9cd5-6fc46ea4dba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:34.380187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf255104-4618-4142-9cd5-6fc46ea4dba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:34.409819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf255104-4618-4142-9cd5-6fc46ea4dba2 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:34.410157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c8b88e7f944005a03d220e4e70c9cb', 'request_id': 'req-44a80adc-6ed2-46cb-b7f0-e0554ea75932'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:34.426158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:34.427707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] The token cache is being invalidate because project c1c8b88e7f944005a03d220e4e70c9cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:37:34.432697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:34.434318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:34.438202 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 854/1722] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:37:33 2022] DELETE /identity/v3/projects/c1c8b88e7f944005a03d220e4e70c9cb => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:34.439486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 869/1723] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:34.442262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:34.442506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf255104-4618-4142-9cd5-6fc46ea4dba2 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:34.445375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:34.555751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a80adc-6ed2-46cb-b7f0-e0554ea75932 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:34.564758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa5155-2ba2-4a59-a5c7-fb71c4509621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:34.565698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa5155-2ba2-4a59-a5c7-fb71c4509621 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:34.565984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa5155-2ba2-4a59-a5c7-fb71c4509621 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:34.566258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa5155-2ba2-4a59-a5c7-fb71c4509621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:34.569748 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ccaa5155-2ba2-4a59-a5c7-fb71c4509621 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:34.583146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf255104-4618-4142-9cd5-6fc46ea4dba2 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:34.600747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:34.601627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:34.603039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:34.603328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] PATH_INFO: `/v3/users/d4aab2437e0a407aa21dd3062d0d72c5` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:34.605820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4aab2437e0a407aa21dd3062d0d72c5)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:34.609004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:34.642117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:34.666607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa5155-2ba2-4a59-a5c7-fb71c4509621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:34.668522 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa5155-2ba2-4a59-a5c7-fb71c4509621 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:34.684231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 855/1724] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:34.868936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4aab2437e0a407aa21dd3062d0d72c5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:34.907338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa5155-2ba2-4a59-a5c7-fb71c4509621 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:34.908994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ccaa5155-2ba2-4a59-a5c7-fb71c4509621 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:34.948707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4aab2437e0a407aa21dd3062d0d72c5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:34.957502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:34.982953 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4aab2437e0a407aa21dd3062d0d72c5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:35.018789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4aab2437e0a407aa21dd3062d0d72c5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:35.024356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4aab2437e0a407aa21dd3062d0d72c5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:35.037513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 870/1725] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:37:34 2022] DELETE /identity/v3/users/d4aab2437e0a407aa21dd3062d0d72c5 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:35.046820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:35.053567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:35.073162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa5155-2ba2-4a59-a5c7-fb71c4509621 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:35.208238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1119a-facf-49cc-adbf-deaa0e5239f9 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:35.222037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d30d8-8584-4be0-a347-a4b017cc20cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:35.222872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d30d8-8584-4be0-a347-a4b017cc20cc service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:35.223201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d30d8-8584-4be0-a347-a4b017cc20cc service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:35.230958 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d30d8-8584-4be0-a347-a4b017cc20cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:35.232702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23caf429-ecda-4758-99ac-1e25e07e4ac1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:35.233717 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23caf429-ecda-4758-99ac-1e25e07e4ac1 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:35.234515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23caf429-ecda-4758-99ac-1e25e07e4ac1 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:35.234802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5e7d30d8-8584-4be0-a347-a4b017cc20cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:35.235161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23caf429-ecda-4758-99ac-1e25e07e4ac1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:35.238613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-23caf429-ecda-4758-99ac-1e25e07e4ac1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:35.270112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d30d8-8584-4be0-a347-a4b017cc20cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:35.276397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d30d8-8584-4be0-a347-a4b017cc20cc service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:35.311763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 871/1726] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:35.324147 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d30d8-8584-4be0-a347-a4b017cc20cc service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:35.326078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5e7d30d8-8584-4be0-a347-a4b017cc20cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:35.448631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23caf429-ecda-4758-99ac-1e25e07e4ac1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:35.455154 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23caf429-ecda-4758-99ac-1e25e07e4ac1 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:35.511755 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 856/1727] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:37:35.512946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23caf429-ecda-4758-99ac-1e25e07e4ac1 service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:35.532808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-23caf429-ecda-4758-99ac-1e25e07e4ac1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:35.562309 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d30d8-8584-4be0-a347-a4b017cc20cc service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:35.576075 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeea89d-88c7-4dd1-9760-2b4d97b9a497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:35.577081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeea89d-88c7-4dd1-9760-2b4d97b9a497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:35.577369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeea89d-88c7-4dd1-9760-2b4d97b9a497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:35.577703 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeea89d-88c7-4dd1-9760-2b4d97b9a497 admin admin] PATH_INFO: `/v3/users/eb1dc96663464c1daf3253d154dacebf` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:35.577826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23caf429-ecda-4758-99ac-1e25e07e4ac1 service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:35.580910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeea89d-88c7-4dd1-9760-2b4d97b9a497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb1dc96663464c1daf3253d154dacebf)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:35.585759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeea89d-88c7-4dd1-9760-2b4d97b9a497 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:35.621265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:35.621265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:35.624851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:35.625925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] PATH_INFO: `/v3/projects/6feca6988aa74d0ca67f74e2b6a93796` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:35.628215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6feca6988aa74d0ca67f74e2b6a93796)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:35.635587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:35.878050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6feca6988aa74d0ca67f74e2b6a93796', 'request_id': 'req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:35.893971 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-baeea89d-88c7-4dd1-9760-2b4d97b9a497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb1dc96663464c1daf3253d154dacebf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:35.894770 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6feca6988aa74d0ca67f74e2b6a93796', 'request_id': 'req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:36.005350 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6feca6988aa74d0ca67f74e2b6a93796', 'request_id': 'req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:36.036562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] The token cache is being invalidate because project 6feca6988aa74d0ca67f74e2b6a93796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:37:36.038873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:36.040153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7db99a9b-6c0f-4e71-a7bd-cc4e2356e97c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:36.044648 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 857/1728] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:37:35 2022] DELETE /identity/v3/projects/6feca6988aa74d0ca67f74e2b6a93796 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:36.049480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f650e-11f6-4739-9722-0c2cfd95866a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:36.050957 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f650e-11f6-4739-9722-0c2cfd95866a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:36.052069 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f650e-11f6-4739-9722-0c2cfd95866a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:36.095113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-639f650e-11f6-4739-9722-0c2cfd95866a None None] MFA Rules not processed for user `966045fbe69f43b4b1a825da3ad571b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:36.096609 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-639f650e-11f6-4739-9722-0c2cfd95866a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:36.112374 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-baeea89d-88c7-4dd1-9760-2b4d97b9a497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb1dc96663464c1daf3253d154dacebf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:36.113340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:36.159090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-baeea89d-88c7-4dd1-9760-2b4d97b9a497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb1dc96663464c1daf3253d154dacebf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:36.188708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-baeea89d-88c7-4dd1-9760-2b4d97b9a497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb1dc96663464c1daf3253d154dacebf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:36.189560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-baeea89d-88c7-4dd1-9760-2b4d97b9a497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb1dc96663464c1daf3253d154dacebf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:36.200908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 872/1729] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:37:35 2022] DELETE /identity/v3/users/eb1dc96663464c1daf3253d154dacebf => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:37:36.202626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446ad59-af58-4739-85da-25af5a116387 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:36.203089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446ad59-af58-4739-85da-25af5a116387 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:36.203909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446ad59-af58-4739-85da-25af5a116387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:36.246315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 858/1730] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:36.248309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f8322-37b7-470b-b49b-da6e3b8d678b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:36.248711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f8322-37b7-470b-b49b-da6e3b8d678b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:36.249078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f8322-37b7-470b-b49b-da6e3b8d678b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:36.261596 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c446ad59-af58-4739-85da-25af5a116387 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:36.271042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c446ad59-af58-4739-85da-25af5a116387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:36.278662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9b9f8322-37b7-470b-b49b-da6e3b8d678b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:36.280203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9b9f8322-37b7-470b-b49b-da6e3b8d678b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:36.554240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 873/1731] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:36.557414 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837666f-b385-40d7-91ba-3f9107889752 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:36.557898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837666f-b385-40d7-91ba-3f9107889752 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:36.564419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837666f-b385-40d7-91ba-3f9107889752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:36.624748 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3837666f-b385-40d7-91ba-3f9107889752 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:36.627463 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3837666f-b385-40d7-91ba-3f9107889752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:36.656414 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 874/1732] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:37:36.658620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837666f-b385-40d7-91ba-3f9107889752 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:36.685166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837666f-b385-40d7-91ba-3f9107889752 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:36.697704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f9c53-5679-45c7-9bde-25f921983de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:36.698742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f9c53-5679-45c7-9bde-25f921983de9 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:36.698969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f9c53-5679-45c7-9bde-25f921983de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:36.699046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f9c53-5679-45c7-9bde-25f921983de9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:36.699855 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 859/1733] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:36.706523 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f9c53-5679-45c7-9bde-25f921983de9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:36.706847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f9c53-5679-45c7-9bde-25f921983de9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:36.708747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f9c53-5679-45c7-9bde-25f921983de9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:36.734775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f8322-37b7-470b-b49b-da6e3b8d678b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:36.737026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9b9f8322-37b7-470b-b49b-da6e3b8d678b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:36.740135 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:36.750013 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 875/1734] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:36.751123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f9c53-5679-45c7-9bde-25f921983de9 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:36.777885 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f9c53-5679-45c7-9bde-25f921983de9 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:36.790350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b753-9ead-497a-b3cd-86a400f78950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:36.791123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b753-9ead-497a-b3cd-86a400f78950 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:36.791470 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b753-9ead-497a-b3cd-86a400f78950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:36.791750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b753-9ead-497a-b3cd-86a400f78950 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:36.792485 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713b753-9ead-497a-b3cd-86a400f78950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:36.794430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713b753-9ead-497a-b3cd-86a400f78950 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:36.906335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f8322-37b7-470b-b49b-da6e3b8d678b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:36.915218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91207b6e-a2e1-4e17-be87-c80b785c4b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:36.915911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207b6e-a2e1-4e17-be87-c80b785c4b13 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:36.916219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207b6e-a2e1-4e17-be87-c80b785c4b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:36.916935 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207b6e-a2e1-4e17-be87-c80b785c4b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:36.945139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91207b6e-a2e1-4e17-be87-c80b785c4b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:36.946750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91207b6e-a2e1-4e17-be87-c80b785c4b13 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:36.967812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 876/1735] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:36 2022] POST /identity/v3/projects => generated 379 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:36.969642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b753-9ead-497a-b3cd-86a400f78950 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:36.970926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 860/1736] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:37.132608 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b753-9ead-497a-b3cd-86a400f78950 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:37.144264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b3140-64d0-4aee-98a0-0d67136fbc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:37.146549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3140-64d0-4aee-98a0-0d67136fbc27 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:37.147174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3140-64d0-4aee-98a0-0d67136fbc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:37.148625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3140-64d0-4aee-98a0-0d67136fbc27 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:37.149369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b3140-64d0-4aee-98a0-0d67136fbc27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:37.158757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b3140-64d0-4aee-98a0-0d67136fbc27 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:37.279564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 877/1737] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:36 2022] POST /identity/v3/users => generated 290 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:37.284506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91207b6e-a2e1-4e17-be87-c80b785c4b13 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:37.304087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91207b6e-a2e1-4e17-be87-c80b785c4b13 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:37.313964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0b21c-e7cb-420c-824c-692a3bc0f3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:37.314707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0b21c-e7cb-420c-824c-692a3bc0f3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:37.315243 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0b21c-e7cb-420c-824c-692a3bc0f3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:37.315584 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0b21c-e7cb-420c-824c-692a3bc0f3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:37.316828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0b21c-e7cb-420c-824c-692a3bc0f3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:37.318883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0b21c-e7cb-420c-824c-692a3bc0f3d9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:37.344136 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 861/1738] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:37.344713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b3140-64d0-4aee-98a0-0d67136fbc27 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:37.355847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b3140-64d0-4aee-98a0-0d67136fbc27 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:37.360755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27a816-fda4-44ac-8dc9-956acc847155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:37.361381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27a816-fda4-44ac-8dc9-956acc847155 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:37.361715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27a816-fda4-44ac-8dc9-956acc847155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:37.362058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27a816-fda4-44ac-8dc9-956acc847155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:37.362788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27a816-fda4-44ac-8dc9-956acc847155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:37.364880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27a816-fda4-44ac-8dc9-956acc847155 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:37.378171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 878/1739] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:37.382277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0b21c-e7cb-420c-824c-692a3bc0f3d9 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:37.394597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0b21c-e7cb-420c-824c-692a3bc0f3d9 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:37.402265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ed45b-de35-4560-ad68-fcaee0a3ce10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:37.403009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ed45b-de35-4560-ad68-fcaee0a3ce10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:37.404567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ed45b-de35-4560-ad68-fcaee0a3ce10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:37.404820 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ed45b-de35-4560-ad68-fcaee0a3ce10 admin admin] PATH_INFO: `/v3/projects/6300799d64944da1af5d5f36b3194d0f/users/a9af61a8e7f543d1964a34c869552147/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:37.424930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ed45b-de35-4560-ad68-fcaee0a3ce10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6300799d64944da1af5d5f36b3194d0f, user_id=a9af61a8e7f543d1964a34c869552147, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:37.439413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ed45b-de35-4560-ad68-fcaee0a3ce10 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:37.545182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 862/1740] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:37 2022] PUT /identity/v3/projects/6300799d64944da1af5d5f36b3194d0f/users/a9af61a8e7f543d1964a34c869552147/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:37.557052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ee92b-9dce-4645-9aa4-bf1c2f3de96c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:37.557629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ee92b-9dce-4645-9aa4-bf1c2f3de96c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:37.557629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ee92b-9dce-4645-9aa4-bf1c2f3de96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:37.605491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-192ee92b-9dce-4645-9aa4-bf1c2f3de96c None None] MFA Rules not processed for user `a9af61a8e7f543d1964a34c869552147`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:37.606750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-192ee92b-9dce-4645-9aa4-bf1c2f3de96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:37.925032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ed45b-de35-4560-ad68-fcaee0a3ce10 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:37.926873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-754ed45b-de35-4560-ad68-fcaee0a3ce10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:37.931721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 879/1741] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:37.933467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ee92b-9dce-4645-9aa4-bf1c2f3de96c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:38.125555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ee92b-9dce-4645-9aa4-bf1c2f3de96c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:38.131525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ed45b-de35-4560-ad68-fcaee0a3ce10 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:38.145724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f50bf-5593-4255-b4d5-5de31f493a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:38.146871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f50bf-5593-4255-b4d5-5de31f493a85 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:38.147152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f50bf-5593-4255-b4d5-5de31f493a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:38.147427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f50bf-5593-4255-b4d5-5de31f493a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:38.148129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f50bf-5593-4255-b4d5-5de31f493a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:38.150745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f50bf-5593-4255-b4d5-5de31f493a85 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:38.165971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:38.165971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:38.165971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:38.165971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] PATH_INFO: `/v3/projects/6f0c260ad9934d7ab8d7e8d54518a74e` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:38.168913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f0c260ad9934d7ab8d7e8d54518a74e)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:38.171193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:38.329574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 880/1742] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:37 2022] POST /identity/v3/projects => generated 381 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:38.338192 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f50bf-5593-4255-b4d5-5de31f493a85 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:38.442621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f0c260ad9934d7ab8d7e8d54518a74e', 'request_id': 'req-de96ecdc-2570-4181-b3be-b1e0708967fd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:38.480416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f0c260ad9934d7ab8d7e8d54518a74e', 'request_id': 'req-de96ecdc-2570-4181-b3be-b1e0708967fd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:38.612243 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f50bf-5593-4255-b4d5-5de31f493a85 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:38.622018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:38.653024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f0c260ad9934d7ab8d7e8d54518a74e', 'request_id': 'req-de96ecdc-2570-4181-b3be-b1e0708967fd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:38.675245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] The token cache is being invalidate because project 6f0c260ad9934d7ab8d7e8d54518a74e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:37:38.675538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:38.681602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-de96ecdc-2570-4181-b3be-b1e0708967fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:38.696007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 863/1743] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:37:37 2022] DELETE /identity/v3/projects/6f0c260ad9934d7ab8d7e8d54518a74e => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:37:38.698125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772852e-1ef4-4fe0-899a-800f0f7c8bb6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:38.710018 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772852e-1ef4-4fe0-899a-800f0f7c8bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:38.710469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772852e-1ef4-4fe0-899a-800f0f7c8bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:38.748058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6772852e-1ef4-4fe0-899a-800f0f7c8bb6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:38.751840 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6772852e-1ef4-4fe0-899a-800f0f7c8bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:38.873794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251bc69-0701-42d4-9ccb-8db138de260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:38.876979 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251bc69-0701-42d4-9ccb-8db138de260c admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:38.879616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251bc69-0701-42d4-9ccb-8db138de260c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:38.880615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251bc69-0701-42d4-9ccb-8db138de260c admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:38.884405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251bc69-0701-42d4-9ccb-8db138de260c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:38.893489 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251bc69-0701-42d4-9ccb-8db138de260c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:38.903571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 864/1744] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:38.910942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1d488-d352-415c-9a25-d3b2e6f99bb4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:38.912187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1d488-d352-415c-9a25-d3b2e6f99bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:38.912916 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1d488-d352-415c-9a25-d3b2e6f99bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:38.976814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-92c1d488-d352-415c-9a25-d3b2e6f99bb4 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:38.978677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-92c1d488-d352-415c-9a25-d3b2e6f99bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:39.016432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 865/1745] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:39.024464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1d488-d352-415c-9a25-d3b2e6f99bb4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:39.036626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1d488-d352-415c-9a25-d3b2e6f99bb4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:39.043164 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40609192-e155-4775-9426-616ef282b66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:39.044221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40609192-e155-4775-9426-616ef282b66c admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:39.044788 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40609192-e155-4775-9426-616ef282b66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:39.045269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40609192-e155-4775-9426-616ef282b66c admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:39.046153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40609192-e155-4775-9426-616ef282b66c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:39.046665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40609192-e155-4775-9426-616ef282b66c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:39.048556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40609192-e155-4775-9426-616ef282b66c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:39.063596 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 881/1746] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:38 2022] POST /identity/v3/users => generated 297 bytes in 730 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:39.071568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 866/1747] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:39.071911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251bc69-0701-42d4-9ccb-8db138de260c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:39.072175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7251bc69-0701-42d4-9ccb-8db138de260c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:39.075223 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40609192-e155-4775-9426-616ef282b66c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:39.118361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40609192-e155-4775-9426-616ef282b66c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:39.140901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251bc69-0701-42d4-9ccb-8db138de260c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:39.144430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c662f9-4484-4754-863b-86163cf56343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:39.145061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c662f9-4484-4754-863b-86163cf56343 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:39.145443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c662f9-4484-4754-863b-86163cf56343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:39.145619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c662f9-4484-4754-863b-86163cf56343 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:39.146266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c662f9-4484-4754-863b-86163cf56343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:39.159691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c662f9-4484-4754-863b-86163cf56343 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:39.166058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9948d-8db1-48cb-8e17-cd13834ce2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:39.166706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9948d-8db1-48cb-8e17-cd13834ce2fe admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:39.168805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9948d-8db1-48cb-8e17-cd13834ce2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:39.169256 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9948d-8db1-48cb-8e17-cd13834ce2fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:39.171237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9948d-8db1-48cb-8e17-cd13834ce2fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:39.173593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9948d-8db1-48cb-8e17-cd13834ce2fe admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:39.229378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 882/1748] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:39.230697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9948d-8db1-48cb-8e17-cd13834ce2fe admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:39.253515 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:39.265272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9948d-8db1-48cb-8e17-cd13834ce2fe admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:39.287927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c63d2-aab0-4cd0-b077-9e601afbc2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:39.288720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c63d2-aab0-4cd0-b077-9e601afbc2bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:39.289077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c63d2-aab0-4cd0-b077-9e601afbc2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:39.289458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c63d2-aab0-4cd0-b077-9e601afbc2bd admin admin] PATH_INFO: `/v3/projects/7664e601c3ca4e5a92b0d17f92ceb8c9/users/df0bb1bf4aed49e99fc341f60b5178d9/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:39.349096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c63d2-aab0-4cd0-b077-9e601afbc2bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7664e601c3ca4e5a92b0d17f92ceb8c9, user_id=df0bb1bf4aed49e99fc341f60b5178d9, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:39.353902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c63d2-aab0-4cd0-b077-9e601afbc2bd admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:39.409520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 867/1749] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:39 2022] POST /identity/v3/projects => generated 371 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:39.418774 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c662f9-4484-4754-863b-86163cf56343 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:39.447941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 883/1750] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:39 2022] PUT /identity/v3/projects/7664e601c3ca4e5a92b0d17f92ceb8c9/users/df0bb1bf4aed49e99fc341f60b5178d9/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:39.451843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c63d2-aab0-4cd0-b077-9e601afbc2bd admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:39.596990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c662f9-4484-4754-863b-86163cf56343 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:39.629594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c63d2-aab0-4cd0-b077-9e601afbc2bd admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:39.642213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8014b-36b7-4cc5-ad33-a95a30472ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:39.643800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8014b-36b7-4cc5-ad33-a95a30472ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:39.644265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8014b-36b7-4cc5-ad33-a95a30472ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:39.644739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8014b-36b7-4cc5-ad33-a95a30472ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:39.645553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8014b-36b7-4cc5-ad33-a95a30472ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:39.647615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8014b-36b7-4cc5-ad33-a95a30472ec7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:39.663412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 884/1751] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:37:39.674117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8014b-36b7-4cc5-ad33-a95a30472ec7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:39.689427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8014b-36b7-4cc5-ad33-a95a30472ec7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:39.695795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fb031-aea2-4d15-9c9c-48727e80e587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:39.697344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb031-aea2-4d15-9c9c-48727e80e587 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:39.697637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb031-aea2-4d15-9c9c-48727e80e587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:39.697927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb031-aea2-4d15-9c9c-48727e80e587 admin admin] PATH_INFO: `/v3/projects/7664e601c3ca4e5a92b0d17f92ceb8c9/users/df0bb1bf4aed49e99fc341f60b5178d9/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:39.707594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fb031-aea2-4d15-9c9c-48727e80e587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7664e601c3ca4e5a92b0d17f92ceb8c9, user_id=df0bb1bf4aed49e99fc341f60b5178d9, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:39.711544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fb031-aea2-4d15-9c9c-48727e80e587 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:39.747939 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 885/1752] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:39 2022] PUT /identity/v3/projects/7664e601c3ca4e5a92b0d17f92ceb8c9/users/df0bb1bf4aed49e99fc341f60b5178d9/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:39.752185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2d826-4f3a-436b-a59a-3b4006bf1168 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:39.752497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2d826-4f3a-436b-a59a-3b4006bf1168 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:39.755694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2d826-4f3a-436b-a59a-3b4006bf1168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:39.802267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132f69d-148e-4e1a-a1af-26d9fc01cec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:39.804076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132f69d-148e-4e1a-a1af-26d9fc01cec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:39.806209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132f69d-148e-4e1a-a1af-26d9fc01cec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:39.807449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132f69d-148e-4e1a-a1af-26d9fc01cec5 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:39.808247 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132f69d-148e-4e1a-a1af-26d9fc01cec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:39.815189 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132f69d-148e-4e1a-a1af-26d9fc01cec5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:39.855650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0bf2d826-4f3a-436b-a59a-3b4006bf1168 None None] MFA Rules not processed for user `df0bb1bf4aed49e99fc341f60b5178d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:39.857846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0bf2d826-4f3a-436b-a59a-3b4006bf1168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:39.947901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 868/1753] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:39 2022] POST /identity/v3/users => generated 286 bytes in 531 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:39.949479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132f69d-148e-4e1a-a1af-26d9fc01cec5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:40.034429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132f69d-148e-4e1a-a1af-26d9fc01cec5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:40.208832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 886/1754] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:40.249030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa459857-9269-4dec-9b92-4836cd29dcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:40.250175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa459857-9269-4dec-9b92-4836cd29dcdd admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:40.250738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa459857-9269-4dec-9b92-4836cd29dcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:40.251258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa459857-9269-4dec-9b92-4836cd29dcdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:40.252169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa459857-9269-4dec-9b92-4836cd29dcdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:40.254392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa459857-9269-4dec-9b92-4836cd29dcdd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:40.271804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 869/1755] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:40.276729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2d826-4f3a-436b-a59a-3b4006bf1168 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:40.314056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2d826-4f3a-436b-a59a-3b4006bf1168 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:40.326959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86bcf2-1980-4de1-aa53-4f0558926d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:40.327589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86bcf2-1980-4de1-aa53-4f0558926d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:40.327867 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86bcf2-1980-4de1-aa53-4f0558926d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:40.328349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86bcf2-1980-4de1-aa53-4f0558926d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:40.329017 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86bcf2-1980-4de1-aa53-4f0558926d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:40.330647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86bcf2-1980-4de1-aa53-4f0558926d5d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:40.350895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 887/1756] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:40.355619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa459857-9269-4dec-9b92-4836cd29dcdd admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:40.371171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa459857-9269-4dec-9b92-4836cd29dcdd admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:40.377690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86bcf2-1980-4de1-aa53-4f0558926d5d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:40.383576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cb86bcf2-1980-4de1-aa53-4f0558926d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:40.410490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e59028-b1a0-4561-80e4-264d96d68734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:40.410490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e59028-b1a0-4561-80e4-264d96d68734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:40.410490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e59028-b1a0-4561-80e4-264d96d68734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:40.410490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e59028-b1a0-4561-80e4-264d96d68734 admin admin] PATH_INFO: `/v3/projects/533d60c92a8d450481f89bbeed2e97b5/users/7134f955e61b45fa8b8377696d3d1fd3/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:40.485668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e59028-b1a0-4561-80e4-264d96d68734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=533d60c92a8d450481f89bbeed2e97b5, user_id=7134f955e61b45fa8b8377696d3d1fd3, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:40.486820 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:40.493277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e59028-b1a0-4561-80e4-264d96d68734 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:40.503752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86bcf2-1980-4de1-aa53-4f0558926d5d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:40.510108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84451a-e696-4507-a7b4-c36c29a98067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:40.510956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84451a-e696-4507-a7b4-c36c29a98067 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:40.513843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84451a-e696-4507-a7b4-c36c29a98067 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:40.514297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84451a-e696-4507-a7b4-c36c29a98067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:40.541215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84451a-e696-4507-a7b4-c36c29a98067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:40.546767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84451a-e696-4507-a7b4-c36c29a98067 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:40.575037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 888/1757] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:40.619552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 870/1758] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:40 2022] PUT /identity/v3/projects/533d60c92a8d450481f89bbeed2e97b5/users/7134f955e61b45fa8b8377696d3d1fd3/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:40.631454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261da8a-9b9a-4a71-9928-9789d4fea2d9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:40.633002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261da8a-9b9a-4a71-9928-9789d4fea2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:40.634282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261da8a-9b9a-4a71-9928-9789d4fea2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:40.738807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7261da8a-9b9a-4a71-9928-9789d4fea2d9 None None] MFA Rules not processed for user `7134f955e61b45fa8b8377696d3d1fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:40.738807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7261da8a-9b9a-4a71-9928-9789d4fea2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:40.855058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 889/1759] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:40.866556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e59028-b1a0-4561-80e4-264d96d68734 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:40.869538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-51e59028-b1a0-4561-80e4-264d96d68734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:40.956630 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:41.050947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e59028-b1a0-4561-80e4-264d96d68734 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:41.063317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243ff6b-e3d0-45ee-a414-255021aa53a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb2129c006f4ddeba7d128a8d43d5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:41.065422 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243ff6b-e3d0-45ee-a414-255021aa53a9 service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:41.069994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243ff6b-e3d0-45ee-a414-255021aa53a9 service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:41.070595 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243ff6b-e3d0-45ee-a414-255021aa53a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:41.104124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243ff6b-e3d0-45ee-a414-255021aa53a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:41.107997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243ff6b-e3d0-45ee-a414-255021aa53a9 service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:41.111439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261da8a-9b9a-4a71-9928-9789d4fea2d9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:41.134417 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 871/1760] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:41.246340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261da8a-9b9a-4a71-9928-9789d4fea2d9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:41.265390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237ff5c-f455-459a-a672-0f5e9cdadc42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:41.266722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237ff5c-f455-459a-a672-0f5e9cdadc42 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:41.267798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237ff5c-f455-459a-a672-0f5e9cdadc42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:41.268388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237ff5c-f455-459a-a672-0f5e9cdadc42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:41.273040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9237ff5c-f455-459a-a672-0f5e9cdadc42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:41.411268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237ff5c-f455-459a-a672-0f5e9cdadc42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:41.413064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237ff5c-f455-459a-a672-0f5e9cdadc42 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:41.425053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 890/1761] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:37:41.576794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0024e-56e6-41ac-8c85-6376c91d401c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:41.577362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0024e-56e6-41ac-8c85-6376c91d401c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:41.577719 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0024e-56e6-41ac-8c85-6376c91d401c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:41.598005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9bb0024e-56e6-41ac-8c85-6376c91d401c None None] MFA Rules not processed for user `05ef5ec6afb84c4abc0cb4862f041dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:41.599366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9bb0024e-56e6-41ac-8c85-6376c91d401c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:41.754495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 872/1762] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:41.781723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060eea8-f276-4e45-99bd-4c50409885ce None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:41.782159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060eea8-f276-4e45-99bd-4c50409885ce None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:41.782301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060eea8-f276-4e45-99bd-4c50409885ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:41.882511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b060eea8-f276-4e45-99bd-4c50409885ce None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:41.884573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b060eea8-f276-4e45-99bd-4c50409885ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:41.896829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:42.024691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 891/1763] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:37:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:42.234973 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:42.444942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0024e-56e6-41ac-8c85-6376c91d401c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:42.446929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9bb0024e-56e6-41ac-8c85-6376c91d401c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:42.604498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0024e-56e6-41ac-8c85-6376c91d401c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:42.611817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2e469-c316-4aed-9f4c-c3052922c2f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:42.612488 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2e469-c316-4aed-9f4c-c3052922c2f9 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:42.612769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2e469-c316-4aed-9f4c-c3052922c2f9 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:42.613046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2e469-c316-4aed-9f4c-c3052922c2f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:42.622989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f2e469-c316-4aed-9f4c-c3052922c2f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:42.624792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f2e469-c316-4aed-9f4c-c3052922c2f9 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:42.635732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad26aa-1f27-44ab-ba4a-42562c3f514e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:42.638166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad26aa-1f27-44ab-ba4a-42562c3f514e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:42.638166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad26aa-1f27-44ab-ba4a-42562c3f514e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:42.638474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 873/1764] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:37:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:37:42.683723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3cad26aa-1f27-44ab-ba4a-42562c3f514e None None] MFA Rules not processed for user `baffd4f6a8ad4a6e91d8d9c61cc1dce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:42.689020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3cad26aa-1f27-44ab-ba4a-42562c3f514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:42.860770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 892/1765] 10.222.0.27 () {58 vars in 1015 bytes} [Sun Aug 7 22:37:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:42.886074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2e469-c316-4aed-9f4c-c3052922c2f9 service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:42.889115 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b7f2e469-c316-4aed-9f4c-c3052922c2f9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:42.918914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2e469-c316-4aed-9f4c-c3052922c2f9 service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:42.924875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b977f3f-13b8-4fc0-b9d0-d6220125ab72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:42.925712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b977f3f-13b8-4fc0-b9d0-d6220125ab72 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:42.926145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b977f3f-13b8-4fc0-b9d0-d6220125ab72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:42.926589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b977f3f-13b8-4fc0-b9d0-d6220125ab72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:42.954794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b977f3f-13b8-4fc0-b9d0-d6220125ab72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:42.958689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b977f3f-13b8-4fc0-b9d0-d6220125ab72 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:42.987836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:43.012915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 874/1766] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:43.739599 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:44.327297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3ef74-6399-4a59-a82f-35e5a4ce0cd2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:44.327297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3ef74-6399-4a59-a82f-35e5a4ce0cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:44.327297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3ef74-6399-4a59-a82f-35e5a4ce0cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:44.383429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3ed3ef74-6399-4a59-a82f-35e5a4ce0cd2 None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:44.391069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3ed3ef74-6399-4a59-a82f-35e5a4ce0cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:44.405432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 893/1767] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:37:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:44.431426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b977f3f-13b8-4fc0-b9d0-d6220125ab72 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:44.445711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b977f3f-13b8-4fc0-b9d0-d6220125ab72 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:44.464635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7f59a-93e2-42cb-97a8-71ce1837f89b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:44.464635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f59a-93e2-42cb-97a8-71ce1837f89b service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:44.464635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f59a-93e2-42cb-97a8-71ce1837f89b service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:44.465754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f59a-93e2-42cb-97a8-71ce1837f89b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:44.486457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7f59a-93e2-42cb-97a8-71ce1837f89b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:44.488427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7f59a-93e2-42cb-97a8-71ce1837f89b service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:44.501816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 875/1768] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:37:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:37:44.652381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:45.424331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cc740-3aeb-4097-9e86-b86b12a48f70 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:45.424856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cc740-3aeb-4097-9e86-b86b12a48f70 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:45.424856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cc740-3aeb-4097-9e86-b86b12a48f70 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:45.426922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 894/1769] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:37:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:37:45.438396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b2bef-acde-42c4-9720-a58f1b343f18 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:45.438994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b2bef-acde-42c4-9720-a58f1b343f18 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:45.438994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b2bef-acde-42c4-9720-a58f1b343f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:45.504265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3d1b2bef-acde-42c4-9720-a58f1b343f18 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:45.509394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3d1b2bef-acde-42c4-9720-a58f1b343f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:45.689648 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 876/1770] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:37:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:45.753593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cc740-3aeb-4097-9e86-b86b12a48f70 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:45.755695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-823cc740-3aeb-4097-9e86-b86b12a48f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:45.795720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cc740-3aeb-4097-9e86-b86b12a48f70 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:45.823800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d2c39-bf10-4b59-bf7c-7d9d31d84803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:45.826912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d2c39-bf10-4b59-bf7c-7d9d31d84803 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:45.827409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d2c39-bf10-4b59-bf7c-7d9d31d84803 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:45.827530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d2c39-bf10-4b59-bf7c-7d9d31d84803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:45.854948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d2c39-bf10-4b59-bf7c-7d9d31d84803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:45.856940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d2c39-bf10-4b59-bf7c-7d9d31d84803 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:45.896242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 895/1771] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:47.311155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b2bef-acde-42c4-9720-a58f1b343f18 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:47.312443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3d1b2bef-acde-42c4-9720-a58f1b343f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:47.486106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b2bef-acde-42c4-9720-a58f1b343f18 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:47.498415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:47.502585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:47.502877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:47.503203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] PATH_INFO: `/v3/users/2270e3b5a8b445d98c29ecb71a3f1927` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:47.510438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2270e3b5a8b445d98c29ecb71a3f1927)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:47.516180 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:47.673657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c268-4744-4204-8287-4be031dc5162 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:47.674316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c268-4744-4204-8287-4be031dc5162 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:47.674316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c268-4744-4204-8287-4be031dc5162 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:47.675941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 896/1772] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:37:47.683941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f8c71-dffb-4519-bdcf-37eef85348c0 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:47.684328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f8c71-dffb-4519-bdcf-37eef85348c0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:47.684548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f8c71-dffb-4519-bdcf-37eef85348c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:47.715513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0c5f8c71-dffb-4519-bdcf-37eef85348c0 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:47.718211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0c5f8c71-dffb-4519-bdcf-37eef85348c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:47.731582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 897/1773] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:37:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:37:47.763916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f8c71-dffb-4519-bdcf-37eef85348c0 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:47.791423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f8c71-dffb-4519-bdcf-37eef85348c0 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:47.801423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab1b7e-b011-44f5-acbd-050cb7d4f7f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:47.802232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab1b7e-b011-44f5-acbd-050cb7d4f7f4 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:47.802648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab1b7e-b011-44f5-acbd-050cb7d4f7f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:47.803203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab1b7e-b011-44f5-acbd-050cb7d4f7f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:47.817413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:47.825711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab1b7e-b011-44f5-acbd-050cb7d4f7f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:47.836127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab1b7e-b011-44f5-acbd-050cb7d4f7f4 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:47.882592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2270e3b5a8b445d98c29ecb71a3f1927'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:47.930734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 898/1774] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:47.942536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2270e3b5a8b445d98c29ecb71a3f1927'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:47.956540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:47.971781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2270e3b5a8b445d98c29ecb71a3f1927'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:47.977673 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2270e3b5a8b445d98c29ecb71a3f1927'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:47.977673 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2270e3b5a8b445d98c29ecb71a3f1927'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:47.998898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 877/1775] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:37:47 2022] DELETE /identity/v3/users/2270e3b5a8b445d98c29ecb71a3f1927 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:48.008311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ee65-9f0b-471f-ae7b-37e95d5fbb12 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:48.008311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ee65-9f0b-471f-ae7b-37e95d5fbb12 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:48.008311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ee65-9f0b-471f-ae7b-37e95d5fbb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:48.122691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-11d4ee65-9f0b-471f-ae7b-37e95d5fbb12 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:48.125471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-11d4ee65-9f0b-471f-ae7b-37e95d5fbb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:48.340724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 899/1776] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:48.472550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:48.472550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:48.549357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091043b-d142-4988-8f51-90a07b03ed5d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:48.553794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9b8f9-0e0b-44b8-8a7b-4bee8e7dc4a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:48.554469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9b8f9-0e0b-44b8-8a7b-4bee8e7dc4a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:48.555287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9b8f9-0e0b-44b8-8a7b-4bee8e7dc4a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:48.555761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9b8f9-0e0b-44b8-8a7b-4bee8e7dc4a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:48.568341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9b8f9-0e0b-44b8-8a7b-4bee8e7dc4a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:48.570021 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9b8f9-0e0b-44b8-8a7b-4bee8e7dc4a2 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:48.581621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 878/1777] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:49.057059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4ee65-9f0b-471f-ae7b-37e95d5fbb12 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:49.064424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-11d4ee65-9f0b-471f-ae7b-37e95d5fbb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:49.072724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4ee65-9f0b-471f-ae7b-37e95d5fbb12 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:49.077483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:49.078115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:49.078390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:49.078662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] PATH_INFO: `/v3/projects/09e9e6e1075c4da8b09ef1358c7a335e` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:49.080961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e9e6e1075c4da8b09ef1358c7a335e)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:49.083891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:49.222829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e9e6e1075c4da8b09ef1358c7a335e', 'request_id': 'req-a149b38d-5161-498a-ad01-349a1a7df96a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:49.230091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e9e6e1075c4da8b09ef1358c7a335e', 'request_id': 'req-a149b38d-5161-498a-ad01-349a1a7df96a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:49.295509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e9e6e1075c4da8b09ef1358c7a335e', 'request_id': 'req-a149b38d-5161-498a-ad01-349a1a7df96a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:49.306779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] The token cache is being invalidate because project 09e9e6e1075c4da8b09ef1358c7a335e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:37:49.307211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:49.307789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a149b38d-5161-498a-ad01-349a1a7df96a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:49.309364 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 900/1778] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:37:49 2022] DELETE /identity/v3/projects/09e9e6e1075c4da8b09ef1358c7a335e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:49.315396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62934d09-7122-4f20-83c4-294e752ce334 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:49.315679 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62934d09-7122-4f20-83c4-294e752ce334 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:49.315958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62934d09-7122-4f20-83c4-294e752ce334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:49.337169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-62934d09-7122-4f20-83c4-294e752ce334 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:49.339250 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-62934d09-7122-4f20-83c4-294e752ce334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:49.399073 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 879/1779] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:49.407132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7ce48-ac6f-463b-81a9-ead96e7569c1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:49.407417 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7ce48-ac6f-463b-81a9-ead96e7569c1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:49.407692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7ce48-ac6f-463b-81a9-ead96e7569c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:49.428210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-fac7ce48-ac6f-463b-81a9-ead96e7569c1 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:49.430108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fac7ce48-ac6f-463b-81a9-ead96e7569c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:49.439670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 901/1780] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:49.443928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62934d09-7122-4f20-83c4-294e752ce334 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:49.453689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62934d09-7122-4f20-83c4-294e752ce334 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:49.458468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737b2ab-aa49-46d2-8f4b-74fc6d6a909a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:49.459160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737b2ab-aa49-46d2-8f4b-74fc6d6a909a admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:49.459432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737b2ab-aa49-46d2-8f4b-74fc6d6a909a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:49.459714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737b2ab-aa49-46d2-8f4b-74fc6d6a909a admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:49.460413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737b2ab-aa49-46d2-8f4b-74fc6d6a909a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:49.460722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737b2ab-aa49-46d2-8f4b-74fc6d6a909a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:49.462309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737b2ab-aa49-46d2-8f4b-74fc6d6a909a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:49.478391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 880/1781] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:49.480339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7ce48-ac6f-463b-81a9-ead96e7569c1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:49.490439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7ce48-ac6f-463b-81a9-ead96e7569c1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:49.495902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce847d01-17eb-40a6-b33b-64f14bd5c105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:49.496721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce847d01-17eb-40a6-b33b-64f14bd5c105 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:49.496995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce847d01-17eb-40a6-b33b-64f14bd5c105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:49.497268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce847d01-17eb-40a6-b33b-64f14bd5c105 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:49.497896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce847d01-17eb-40a6-b33b-64f14bd5c105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:49.499835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce847d01-17eb-40a6-b33b-64f14bd5c105 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:49.596178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 902/1782] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:49 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:49.602136 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737b2ab-aa49-46d2-8f4b-74fc6d6a909a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:49.613429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737b2ab-aa49-46d2-8f4b-74fc6d6a909a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:49.675640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba928f94-fe9c-4086-a216-a3a3468b0ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:49.676364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba928f94-fe9c-4086-a216-a3a3468b0ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:49.678818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba928f94-fe9c-4086-a216-a3a3468b0ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:49.678818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba928f94-fe9c-4086-a216-a3a3468b0ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:49.678818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba928f94-fe9c-4086-a216-a3a3468b0ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:49.680338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba928f94-fe9c-4086-a216-a3a3468b0ca8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:49.764967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:49.770948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 881/1783] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:49 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:49.775916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce847d01-17eb-40a6-b33b-64f14bd5c105 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:49.786213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce847d01-17eb-40a6-b33b-64f14bd5c105 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:49.791618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900f803-77b4-4fb5-a1a0-0e20b072bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:49.792259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900f803-77b4-4fb5-a1a0-0e20b072bd5f admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:49.792552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900f803-77b4-4fb5-a1a0-0e20b072bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:49.792811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900f803-77b4-4fb5-a1a0-0e20b072bd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:49.793460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900f803-77b4-4fb5-a1a0-0e20b072bd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:49.795055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900f803-77b4-4fb5-a1a0-0e20b072bd5f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:49.805256 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 903/1784] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:49.809199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba928f94-fe9c-4086-a216-a3a3468b0ca8 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:49.818997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba928f94-fe9c-4086-a216-a3a3468b0ca8 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:49.823844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffa94b-521d-4ab4-92f8-bd7bbb9b2cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:49.824473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffa94b-521d-4ab4-92f8-bd7bbb9b2cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:49.824755 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffa94b-521d-4ab4-92f8-bd7bbb9b2cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:49.825027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffa94b-521d-4ab4-92f8-bd7bbb9b2cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:49.825668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ffa94b-521d-4ab4-92f8-bd7bbb9b2cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:49.827253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ffa94b-521d-4ab4-92f8-bd7bbb9b2cb0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:49.838600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 882/1785] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:49.842829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900f803-77b4-4fb5-a1a0-0e20b072bd5f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:49.853162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900f803-77b4-4fb5-a1a0-0e20b072bd5f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:49.858692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483924c3-1431-41be-9389-6b6bbd6a1521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:49.858692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483924c3-1431-41be-9389-6b6bbd6a1521 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:49.859171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483924c3-1431-41be-9389-6b6bbd6a1521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:49.859329 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483924c3-1431-41be-9389-6b6bbd6a1521 admin admin] PATH_INFO: `/v3/projects/f97ffb14eeef44ca933d81859e8ba339/users/991e58aedbb1449fb5e1cff5ae04fefc/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:49.877495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483924c3-1431-41be-9389-6b6bbd6a1521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97ffb14eeef44ca933d81859e8ba339, user_id=991e58aedbb1449fb5e1cff5ae04fefc, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:49.881676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483924c3-1431-41be-9389-6b6bbd6a1521 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:49.953951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 904/1786] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:49 2022] PUT /identity/v3/projects/f97ffb14eeef44ca933d81859e8ba339/users/991e58aedbb1449fb5e1cff5ae04fefc/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:37:49.964498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6ad1d-6c6d-44a3-a630-e5e68e058f31 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:49.964784 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6ad1d-6c6d-44a3-a630-e5e68e058f31 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:49.965059 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6ad1d-6c6d-44a3-a630-e5e68e058f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:49.999360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-63e6ad1d-6c6d-44a3-a630-e5e68e058f31 None None] MFA Rules not processed for user `991e58aedbb1449fb5e1cff5ae04fefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:50.000604 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-63e6ad1d-6c6d-44a3-a630-e5e68e058f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:50.065472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 883/1787] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:50.076756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483924c3-1431-41be-9389-6b6bbd6a1521 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:50.078477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-483924c3-1431-41be-9389-6b6bbd6a1521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:50.092866 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:50.137667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483924c3-1431-41be-9389-6b6bbd6a1521 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:50.141377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b84ce-c874-44d0-91ab-a8c45346fc2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:50.142018 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b84ce-c874-44d0-91ab-a8c45346fc2d service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:50.142295 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b84ce-c874-44d0-91ab-a8c45346fc2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:50.142564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b84ce-c874-44d0-91ab-a8c45346fc2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:50.151614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b84ce-c874-44d0-91ab-a8c45346fc2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:50.152954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b84ce-c874-44d0-91ab-a8c45346fc2d service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:50.161587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 905/1788] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:50.184330 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6ad1d-6c6d-44a3-a630-e5e68e058f31 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:50.249701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6ad1d-6c6d-44a3-a630-e5e68e058f31 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:50.255169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:50.255810 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:50.256122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:50.256454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] PATH_INFO: `/v3/users/991e58aedbb1449fb5e1cff5ae04fefc` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:50.257951 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=991e58aedbb1449fb5e1cff5ae04fefc)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:50.260084 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:50.406789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991e58aedbb1449fb5e1cff5ae04fefc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:50.467418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991e58aedbb1449fb5e1cff5ae04fefc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:50.481313 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991e58aedbb1449fb5e1cff5ae04fefc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:50.482031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991e58aedbb1449fb5e1cff5ae04fefc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:50.482643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991e58aedbb1449fb5e1cff5ae04fefc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:50.494661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 884/1789] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:37:50 2022] DELETE /identity/v3/users/991e58aedbb1449fb5e1cff5ae04fefc => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:50.499365 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39444d31-eb9c-4547-ba64-9c9642046c91 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:50.499646 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39444d31-eb9c-4547-ba64-9c9642046c91 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:50.499948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39444d31-eb9c-4547-ba64-9c9642046c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:50.519014 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-39444d31-eb9c-4547-ba64-9c9642046c91 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:50.520276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-39444d31-eb9c-4547-ba64-9c9642046c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:50.584674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 906/1790] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:50.594690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:50.596221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:50.659299 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fed597-fa18-42ed-9461-0973a9863a30 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:50.663126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768094d6-de24-462f-9474-b792e1b4d3cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:50.664459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768094d6-de24-462f-9474-b792e1b4d3cb service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:50.664750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768094d6-de24-462f-9474-b792e1b4d3cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:50.665023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768094d6-de24-462f-9474-b792e1b4d3cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:50.669176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:50.677297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768094d6-de24-462f-9474-b792e1b4d3cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:50.678733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768094d6-de24-462f-9474-b792e1b4d3cb service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:50.691969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 885/1791] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:37:51.299409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39444d31-eb9c-4547-ba64-9c9642046c91 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:51.303327 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-39444d31-eb9c-4547-ba64-9c9642046c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:51.324795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39444d31-eb9c-4547-ba64-9c9642046c91 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:51.326764 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:51.328235 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:51.328876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:51.329345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] PATH_INFO: `/v3/projects/f97ffb14eeef44ca933d81859e8ba339` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:51.330857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f97ffb14eeef44ca933d81859e8ba339)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:51.333345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:51.490091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f97ffb14eeef44ca933d81859e8ba339', 'request_id': 'req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:51.498166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f97ffb14eeef44ca933d81859e8ba339', 'request_id': 'req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:51.541766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f97ffb14eeef44ca933d81859e8ba339', 'request_id': 'req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:51.556003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] The token cache is being invalidate because project f97ffb14eeef44ca933d81859e8ba339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:37:51.556342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:51.557118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be7a4aa4-aa62-459f-ab91-b76fe27aeff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:37:51.559036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 907/1792] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:37:51 2022] DELETE /identity/v3/projects/f97ffb14eeef44ca933d81859e8ba339 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:51.565608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e2e7e-c4b7-494e-9b2d-d6baa556948f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:51.566206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e2e7e-c4b7-494e-9b2d-d6baa556948f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:51.566871 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e2e7e-c4b7-494e-9b2d-d6baa556948f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:51.617949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-369e2e7e-c4b7-494e-9b2d-d6baa556948f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:51.617949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-369e2e7e-c4b7-494e-9b2d-d6baa556948f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:51.707694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 886/1793] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:51.717534 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d03521-fcab-41ed-959d-b59e15a3a5e9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:51.717820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d03521-fcab-41ed-959d-b59e15a3a5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:51.718096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d03521-fcab-41ed-959d-b59e15a3a5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:51.738960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-98d03521-fcab-41ed-959d-b59e15a3a5e9 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:51.740272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-98d03521-fcab-41ed-959d-b59e15a3a5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:51.754312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e2e7e-c4b7-494e-9b2d-d6baa556948f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:51.754858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 908/1794] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:37:51.767252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e2e7e-c4b7-494e-9b2d-d6baa556948f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:51.772651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bf7d5-a69b-48d1-b1b3-ca01f3383118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:51.785564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bf7d5-a69b-48d1-b1b3-ca01f3383118 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:51.785564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bf7d5-a69b-48d1-b1b3-ca01f3383118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:51.785564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bf7d5-a69b-48d1-b1b3-ca01f3383118 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:51.785564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bf7d5-a69b-48d1-b1b3-ca01f3383118 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:37:51.785564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bf7d5-a69b-48d1-b1b3-ca01f3383118 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:51.785564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bf7d5-a69b-48d1-b1b3-ca01f3383118 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:51.795194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 887/1795] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:37:51.799873 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d03521-fcab-41ed-959d-b59e15a3a5e9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:51.812055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d03521-fcab-41ed-959d-b59e15a3a5e9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:51.818989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b39351-f88c-44bb-bea7-5951dd9db9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:51.819702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b39351-f88c-44bb-bea7-5951dd9db9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:51.825533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b39351-f88c-44bb-bea7-5951dd9db9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:51.825848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b39351-f88c-44bb-bea7-5951dd9db9b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:51.826527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b39351-f88c-44bb-bea7-5951dd9db9b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:51.828655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b39351-f88c-44bb-bea7-5951dd9db9b8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:52.000056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 909/1796] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:37:51 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:37:52.001714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bf7d5-a69b-48d1-b1b3-ca01f3383118 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:52.015307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bf7d5-a69b-48d1-b1b3-ca01f3383118 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:52.121426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb78390-58cf-43e7-9be7-6505b5a8d266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:52.121958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb78390-58cf-43e7-9be7-6505b5a8d266 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:52.122240 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb78390-58cf-43e7-9be7-6505b5a8d266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:52.126103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb78390-58cf-43e7-9be7-6505b5a8d266 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:52.126978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb78390-58cf-43e7-9be7-6505b5a8d266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:52.129645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb78390-58cf-43e7-9be7-6505b5a8d266 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:52.226668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 888/1797] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:37:52 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:37:52.227630 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b39351-f88c-44bb-bea7-5951dd9db9b8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:52.264806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b39351-f88c-44bb-bea7-5951dd9db9b8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:52.270223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738dc7f-8b94-4a01-a6bf-3169090162ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:52.276300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738dc7f-8b94-4a01-a6bf-3169090162ae admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:52.276792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738dc7f-8b94-4a01-a6bf-3169090162ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:52.278229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738dc7f-8b94-4a01-a6bf-3169090162ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:52.279070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738dc7f-8b94-4a01-a6bf-3169090162ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:52.282143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738dc7f-8b94-4a01-a6bf-3169090162ae admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:52.293872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:52.301695 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb78390-58cf-43e7-9be7-6505b5a8d266 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:52.302698 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 910/1798] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:52.330550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb78390-58cf-43e7-9be7-6505b5a8d266 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:52.342936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d3d98-c912-4ac8-bea1-c58ff58981dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:52.343752 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d3d98-c912-4ac8-bea1-c58ff58981dc admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:52.344172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d3d98-c912-4ac8-bea1-c58ff58981dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:52.350548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d3d98-c912-4ac8-bea1-c58ff58981dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:52.350548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d3d98-c912-4ac8-bea1-c58ff58981dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:52.350548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d3d98-c912-4ac8-bea1-c58ff58981dc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:52.368425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738dc7f-8b94-4a01-a6bf-3169090162ae admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:52.383858 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 889/1799] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:37:52.384208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738dc7f-8b94-4a01-a6bf-3169090162ae admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:52.398019 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73956e40-c22d-4283-8b78-f4070e1e5806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:52.399000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73956e40-c22d-4283-8b78-f4070e1e5806 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:52.399278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73956e40-c22d-4283-8b78-f4070e1e5806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:52.399554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73956e40-c22d-4283-8b78-f4070e1e5806 admin admin] PATH_INFO: `/v3/projects/b63c1bc8d599429d85636830b6de0e2a/users/e936defbdd6c42dc852b12401b23ea16/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:52.418088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73956e40-c22d-4283-8b78-f4070e1e5806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63c1bc8d599429d85636830b6de0e2a, user_id=e936defbdd6c42dc852b12401b23ea16, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:52.422194 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73956e40-c22d-4283-8b78-f4070e1e5806 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:52.697207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 911/1800] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:37:52 2022] PUT /identity/v3/projects/b63c1bc8d599429d85636830b6de0e2a/users/e936defbdd6c42dc852b12401b23ea16/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:37:52.707669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16113c31-6950-4283-b40a-a60ba7482815 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:52.708376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16113c31-6950-4283-b40a-a60ba7482815 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:52.708376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16113c31-6950-4283-b40a-a60ba7482815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:52.717668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:37:52.757612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-16113c31-6950-4283-b40a-a60ba7482815 None None] MFA Rules not processed for user `e936defbdd6c42dc852b12401b23ea16`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:37:52.759423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-16113c31-6950-4283-b40a-a60ba7482815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:52.904140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 890/1801] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:37:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:37:53.019856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73956e40-c22d-4283-8b78-f4070e1e5806 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:37:53.021896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-73956e40-c22d-4283-8b78-f4070e1e5806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:37:53.135486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73956e40-c22d-4283-8b78-f4070e1e5806 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:37:53.135486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e1e1a-21af-4ea0-b73a-3d7e1c8591c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:37:53.135486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e1e1a-21af-4ea0-b73a-3d7e1c8591c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:37:53.135486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e1e1a-21af-4ea0-b73a-3d7e1c8591c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:37:53.135875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e1e1a-21af-4ea0-b73a-3d7e1c8591c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:37:53.142434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e1e1a-21af-4ea0-b73a-3d7e1c8591c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:37:53.144246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e1e1a-21af-4ea0-b73a-3d7e1c8591c8 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:37:53.181267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 912/1802] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:37:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:03.250124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:03.324383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=30 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:06.424946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16113c31-6950-4283-b40a-a60ba7482815 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:06.426588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-16113c31-6950-4283-b40a-a60ba7482815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:06.496718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16113c31-6950-4283-b40a-a60ba7482815 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:06.501428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:06.502064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:06.502346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:06.502623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] PATH_INFO: `/v3/users/a9af61a8e7f543d1964a34c869552147` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:06.503772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9af61a8e7f543d1964a34c869552147)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:06.505769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:06.680952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9af61a8e7f543d1964a34c869552147'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:06.740927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9af61a8e7f543d1964a34c869552147'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:06.755841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9af61a8e7f543d1964a34c869552147'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:06.756801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9af61a8e7f543d1964a34c869552147'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:06.757792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9af61a8e7f543d1964a34c869552147'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:06.777752 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 891/1803] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:06 2022] DELETE /identity/v3/users/a9af61a8e7f543d1964a34c869552147 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:06.782306 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec54f8-4c2b-44ca-8fba-f995466182be None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:06.782809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec54f8-4c2b-44ca-8fba-f995466182be None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:06.782898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec54f8-4c2b-44ca-8fba-f995466182be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:06.802703 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-80ec54f8-4c2b-44ca-8fba-f995466182be None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:06.803909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-80ec54f8-4c2b-44ca-8fba-f995466182be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:06.898168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 913/1804] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:06.912384 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:06.913771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:06.939990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec54f8-4c2b-44ca-8fba-f995466182be None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:07.164386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-80ec54f8-4c2b-44ca-8fba-f995466182be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:07.165016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fee93-45a3-4be0-8a37-7d6aa61c944d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:07.165016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:07.165016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:07.165016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:07.165546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] PATH_INFO: `/v3/users/e936defbdd6c42dc852b12401b23ea16` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:07.165546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] RBAC: Authorizing `identity:delete_user(user_id=e936defbdd6c42dc852b12401b23ea16)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:07.165546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:07.165850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec54f8-4c2b-44ca-8fba-f995466182be None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:07.165850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5792e6-b291-43b1-8596-92dc7d2c07fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:07.165850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5792e6-b291-43b1-8596-92dc7d2c07fe service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:07.165850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5792e6-b291-43b1-8596-92dc7d2c07fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:07.166361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5792e6-b291-43b1-8596-92dc7d2c07fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:07.166361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5792e6-b291-43b1-8596-92dc7d2c07fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:07.166361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5792e6-b291-43b1-8596-92dc7d2c07fe service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:07.166361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 914/1805] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:07.329633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e936defbdd6c42dc852b12401b23ea16'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:07.431818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e936defbdd6c42dc852b12401b23ea16'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:07.456735 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e936defbdd6c42dc852b12401b23ea16'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:07.457627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e936defbdd6c42dc852b12401b23ea16'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:07.458454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e936defbdd6c42dc852b12401b23ea16'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:07.460968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:07.474620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 892/1806] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:06 2022] DELETE /identity/v3/users/e936defbdd6c42dc852b12401b23ea16 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:07.480459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11809a11-b848-4d27-a4bc-833ac3022ee7 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:07.480744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11809a11-b848-4d27-a4bc-833ac3022ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:07.481019 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11809a11-b848-4d27-a4bc-833ac3022ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:07.511978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-11809a11-b848-4d27-a4bc-833ac3022ee7 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:07.513611 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-11809a11-b848-4d27-a4bc-833ac3022ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:07.613125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 915/1807] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:07.623966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:07.625634 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:07.692731 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532d60e-2665-48c8-8135-d557989eebff admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:07.697831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4db3e-5c12-48a3-8c57-6e882c1e0e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:07.698725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4db3e-5c12-48a3-8c57-6e882c1e0e9e service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:07.699135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4db3e-5c12-48a3-8c57-6e882c1e0e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:07.699539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4db3e-5c12-48a3-8c57-6e882c1e0e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:07.710063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4db3e-5c12-48a3-8c57-6e882c1e0e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:07.711806 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4db3e-5c12-48a3-8c57-6e882c1e0e9e service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:07.726138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 893/1808] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:08.406619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11809a11-b848-4d27-a4bc-833ac3022ee7 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:08.408128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-11809a11-b848-4d27-a4bc-833ac3022ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:08.434538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11809a11-b848-4d27-a4bc-833ac3022ee7 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:08.441431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:08.442059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:08.442586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:08.442963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] PATH_INFO: `/v3/projects/6300799d64944da1af5d5f36b3194d0f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:08.445029 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6300799d64944da1af5d5f36b3194d0f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:08.448007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:08.472656 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4db3e-5c12-48a3-8c57-6e882c1e0e9e service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:08.475596 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f2d4db3e-5c12-48a3-8c57-6e882c1e0e9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:08.503446 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4db3e-5c12-48a3-8c57-6e882c1e0e9e service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:08.517356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:08.518159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:08.518539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:08.519055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] PATH_INFO: `/v3/projects/b63c1bc8d599429d85636830b6de0e2a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:08.521674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63c1bc8d599429d85636830b6de0e2a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:08.525525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:08.653284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:08.669748 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6300799d64944da1af5d5f36b3194d0f', 'request_id': 'req-3869af1d-d993-442d-84ab-8fce8ab35adc'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:08.670078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63c1bc8d599429d85636830b6de0e2a', 'request_id': 'req-efd35e26-0740-4423-bb7f-efd76ff8ec52'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:08.677400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6300799d64944da1af5d5f36b3194d0f', 'request_id': 'req-3869af1d-d993-442d-84ab-8fce8ab35adc'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:08.678582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63c1bc8d599429d85636830b6de0e2a', 'request_id': 'req-efd35e26-0740-4423-bb7f-efd76ff8ec52'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:08.742037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6300799d64944da1af5d5f36b3194d0f', 'request_id': 'req-3869af1d-d993-442d-84ab-8fce8ab35adc'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:08.765556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] The token cache is being invalidate because project 6300799d64944da1af5d5f36b3194d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:08.766086 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:08.767627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:08.772657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 916/1809] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:08 2022] DELETE /identity/v3/projects/6300799d64944da1af5d5f36b3194d0f => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:08.772657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:08.776131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:08.827060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63c1bc8d599429d85636830b6de0e2a', 'request_id': 'req-efd35e26-0740-4423-bb7f-efd76ff8ec52'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:08.843411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] The token cache is being invalidate because project b63c1bc8d599429d85636830b6de0e2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:08.843844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:08.848507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:08.850377 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 894/1810] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:08 2022] DELETE /identity/v3/projects/b63c1bc8d599429d85636830b6de0e2a => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:08.851334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:08.856900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:08.884637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869af1d-d993-442d-84ab-8fce8ab35adc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:09.012495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd35e26-0740-4423-bb7f-efd76ff8ec52 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:09.014661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d0d34-e134-4b4a-8f45-e598c3e74fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:09.015425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d0d34-e134-4b4a-8f45-e598c3e74fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:09.015715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d0d34-e134-4b4a-8f45-e598c3e74fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:09.016005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d0d34-e134-4b4a-8f45-e598c3e74fb5 admin admin] PATH_INFO: `/v3/users/52d15ab0003c4b9e86c614bf18b7700e` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:09.017284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d0d34-e134-4b4a-8f45-e598c3e74fb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d15ab0003c4b9e86c614bf18b7700e)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:09.019072 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bb7ac-5813-4eaf-bed2-f6d1879b8353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:09.019595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d0d34-e134-4b4a-8f45-e598c3e74fb5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:09.020050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bb7ac-5813-4eaf-bed2-f6d1879b8353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:09.024988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bb7ac-5813-4eaf-bed2-f6d1879b8353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:09.024988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bb7ac-5813-4eaf-bed2-f6d1879b8353 admin admin] PATH_INFO: `/v3/users/df0bb1bf4aed49e99fc341f60b5178d9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:09.024988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bb7ac-5813-4eaf-bed2-f6d1879b8353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0bb1bf4aed49e99fc341f60b5178d9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:09.027755 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bb7ac-5813-4eaf-bed2-f6d1879b8353 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:09.061503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:09.273258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0f9bb7ac-5813-4eaf-bed2-f6d1879b8353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0bb1bf4aed49e99fc341f60b5178d9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:09.298859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2c0d0d34-e134-4b4a-8f45-e598c3e74fb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d15ab0003c4b9e86c614bf18b7700e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:09.376958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0f9bb7ac-5813-4eaf-bed2-f6d1879b8353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0bb1bf4aed49e99fc341f60b5178d9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:09.394639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0f9bb7ac-5813-4eaf-bed2-f6d1879b8353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0bb1bf4aed49e99fc341f60b5178d9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:09.395694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0f9bb7ac-5813-4eaf-bed2-f6d1879b8353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0bb1bf4aed49e99fc341f60b5178d9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:09.395694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0f9bb7ac-5813-4eaf-bed2-f6d1879b8353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0bb1bf4aed49e99fc341f60b5178d9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:09.410014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 895/1811] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:08 2022] DELETE /identity/v3/users/df0bb1bf4aed49e99fc341f60b5178d9 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:09.411527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abb006-a7b3-4497-9041-80a5b1666007 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:09.412093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abb006-a7b3-4497-9041-80a5b1666007 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:09.412321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abb006-a7b3-4497-9041-80a5b1666007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:09.446868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-11abb006-a7b3-4497-9041-80a5b1666007 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:09.448169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-11abb006-a7b3-4497-9041-80a5b1666007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:09.489938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2c0d0d34-e134-4b4a-8f45-e598c3e74fb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d15ab0003c4b9e86c614bf18b7700e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:09.508552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2c0d0d34-e134-4b4a-8f45-e598c3e74fb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d15ab0003c4b9e86c614bf18b7700e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:09.514045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:09.518800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2c0d0d34-e134-4b4a-8f45-e598c3e74fb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d15ab0003c4b9e86c614bf18b7700e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:09.519871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2c0d0d34-e134-4b4a-8f45-e598c3e74fb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d15ab0003c4b9e86c614bf18b7700e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:09.529545 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 917/1812] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:08 2022] DELETE /identity/v3/users/52d15ab0003c4b9e86c614bf18b7700e => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:09.533162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3ab62-1ab6-46fc-9ade-7e92fac55af1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:09.533809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3ab62-1ab6-46fc-9ade-7e92fac55af1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:09.533938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 896/1813] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:09.534694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3ab62-1ab6-46fc-9ade-7e92fac55af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:09.535356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b723c88-80f2-4c02-9230-8d17d7bb06c4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:09.536303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b723c88-80f2-4c02-9230-8d17d7bb06c4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:09.536590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b723c88-80f2-4c02-9230-8d17d7bb06c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:09.559378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-5b723c88-80f2-4c02-9230-8d17d7bb06c4 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:09.560618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5b723c88-80f2-4c02-9230-8d17d7bb06c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:09.561298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-61d3ab62-1ab6-46fc-9ade-7e92fac55af1 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:09.562761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-61d3ab62-1ab6-46fc-9ade-7e92fac55af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:09.638880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 897/1814] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:09.652059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 918/1815] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:09.654671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b723c88-80f2-4c02-9230-8d17d7bb06c4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:09.677272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3ab62-1ab6-46fc-9ade-7e92fac55af1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:09.678515 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-61d3ab62-1ab6-46fc-9ade-7e92fac55af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:09.694599 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b723c88-80f2-4c02-9230-8d17d7bb06c4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:09.704731 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753acd2-1152-4090-bef8-3196a61fdef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:09.706161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753acd2-1152-4090-bef8-3196a61fdef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:09.706601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753acd2-1152-4090-bef8-3196a61fdef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:09.707021 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753acd2-1152-4090-bef8-3196a61fdef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:09.708030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753acd2-1152-4090-bef8-3196a61fdef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:38:09.708500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753acd2-1152-4090-bef8-3196a61fdef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:09.710519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753acd2-1152-4090-bef8-3196a61fdef2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:09.731625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 898/1816] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:38:09.732769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753acd2-1152-4090-bef8-3196a61fdef2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:09.744024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753acd2-1152-4090-bef8-3196a61fdef2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:09.749383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385164d-d5c0-482d-98f1-ac7ab0c628e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:09.750050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385164d-d5c0-482d-98f1-ac7ab0c628e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:09.750335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385164d-d5c0-482d-98f1-ac7ab0c628e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:09.750608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385164d-d5c0-482d-98f1-ac7ab0c628e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:09.751824 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385164d-d5c0-482d-98f1-ac7ab0c628e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:09.754131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385164d-d5c0-482d-98f1-ac7ab0c628e6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:09.778426 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3ab62-1ab6-46fc-9ade-7e92fac55af1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:09.782540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2b0e5-61dc-4d29-93f8-cc90c91a51b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:09.783232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2b0e5-61dc-4d29-93f8-cc90c91a51b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:09.783579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2b0e5-61dc-4d29-93f8-cc90c91a51b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:09.783923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2b0e5-61dc-4d29-93f8-cc90c91a51b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:09.795392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2b0e5-61dc-4d29-93f8-cc90c91a51b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:09.797423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2b0e5-61dc-4d29-93f8-cc90c91a51b2 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:09.817140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 919/1817] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:38:09.890419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 899/1818] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:09 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:09.894893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2b0e5-61dc-4d29-93f8-cc90c91a51b2 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:09.912824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2b0e5-61dc-4d29-93f8-cc90c91a51b2 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:09.913851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385164d-d5c0-482d-98f1-ac7ab0c628e6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:09.916206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1385164d-d5c0-482d-98f1-ac7ab0c628e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:09.924906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:09.996638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31f678-f213-4196-a953-2237db80cfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:09.997446 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31f678-f213-4196-a953-2237db80cfb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:09.997731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31f678-f213-4196-a953-2237db80cfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:09.998233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31f678-f213-4196-a953-2237db80cfb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:09.998797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31f678-f213-4196-a953-2237db80cfb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:10.001448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31f678-f213-4196-a953-2237db80cfb1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:10.082971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385164d-d5c0-482d-98f1-ac7ab0c628e6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:10.092352 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:10.093646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:10.095083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:10.097184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] PATH_INFO: `/v3/projects/7664e601c3ca4e5a92b0d17f92ceb8c9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:10.099925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7664e601c3ca4e5a92b0d17f92ceb8c9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:10.102137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:10.135711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 920/1819] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:09 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:10.137347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31f678-f213-4196-a953-2237db80cfb1 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:10.154348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31f678-f213-4196-a953-2237db80cfb1 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:10.160304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb9ed2-6274-4c80-aaf6-58ef2a4da908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:10.161366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb9ed2-6274-4c80-aaf6-58ef2a4da908 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:10.161366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb9ed2-6274-4c80-aaf6-58ef2a4da908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:10.161648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb9ed2-6274-4c80-aaf6-58ef2a4da908 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:10.162319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb9ed2-6274-4c80-aaf6-58ef2a4da908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:10.164472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb9ed2-6274-4c80-aaf6-58ef2a4da908 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:10.194901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 921/1820] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:10.220051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb9ed2-6274-4c80-aaf6-58ef2a4da908 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:10.231309 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb9ed2-6274-4c80-aaf6-58ef2a4da908 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:10.238507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b30ee-88e6-45fe-b37e-e461cf683ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:10.239144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b30ee-88e6-45fe-b37e-e461cf683ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:10.239418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b30ee-88e6-45fe-b37e-e461cf683ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:10.239693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b30ee-88e6-45fe-b37e-e461cf683ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:10.244701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b30ee-88e6-45fe-b37e-e461cf683ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:10.246404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b30ee-88e6-45fe-b37e-e461cf683ec3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:10.279976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 922/1821] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:10.288343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b30ee-88e6-45fe-b37e-e461cf683ec3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:10.294513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7664e601c3ca4e5a92b0d17f92ceb8c9', 'request_id': 'req-75b63c81-a611-4102-a34c-edfaff7e35d0'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:10.302111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b30ee-88e6-45fe-b37e-e461cf683ec3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:10.304270 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7664e601c3ca4e5a92b0d17f92ceb8c9', 'request_id': 'req-75b63c81-a611-4102-a34c-edfaff7e35d0'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:10.386856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5316d3-ea14-44a5-b1ce-7402c1b30149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:10.388504 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7664e601c3ca4e5a92b0d17f92ceb8c9', 'request_id': 'req-75b63c81-a611-4102-a34c-edfaff7e35d0'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:10.388905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5316d3-ea14-44a5-b1ce-7402c1b30149 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:10.389467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5316d3-ea14-44a5-b1ce-7402c1b30149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:10.390184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5316d3-ea14-44a5-b1ce-7402c1b30149 admin admin] PATH_INFO: `/v3/projects/e36146acdd4946399eb3bce115af6b1c/users/90bad0adb01a4d34876826ac6fa18551/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:10.408275 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] The token cache is being invalidate because project 7664e601c3ca4e5a92b0d17f92ceb8c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:10.408507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:10.410142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-75b63c81-a611-4102-a34c-edfaff7e35d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:10.416191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 900/1822] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:09 2022] DELETE /identity/v3/projects/7664e601c3ca4e5a92b0d17f92ceb8c9 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:10.425849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93534376-83f7-423c-9a82-7a3ddb8f8cc8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:10.425849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93534376-83f7-423c-9a82-7a3ddb8f8cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:10.425849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93534376-83f7-423c-9a82-7a3ddb8f8cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:10.426664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5316d3-ea14-44a5-b1ce-7402c1b30149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e36146acdd4946399eb3bce115af6b1c, user_id=90bad0adb01a4d34876826ac6fa18551, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:10.439167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5316d3-ea14-44a5-b1ce-7402c1b30149 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:10.508898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-93534376-83f7-423c-9a82-7a3ddb8f8cc8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:10.510175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-93534376-83f7-423c-9a82-7a3ddb8f8cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:10.534685 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 923/1823] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:10 2022] PUT /identity/v3/projects/e36146acdd4946399eb3bce115af6b1c/users/90bad0adb01a4d34876826ac6fa18551/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:10.543593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 901/1824] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:10.543593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff598-cde2-4771-a307-b1aa794d0e0e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:10.543593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff598-cde2-4771-a307-b1aa794d0e0e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:10.543593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff598-cde2-4771-a307-b1aa794d0e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:10.547257 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ba429-fbc8-414b-bfb0-834c1c92ecaa None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:10.547745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ba429-fbc8-414b-bfb0-834c1c92ecaa None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:10.548207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ba429-fbc8-414b-bfb0-834c1c92ecaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:10.581557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:10.583879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1d7ba429-fbc8-414b-bfb0-834c1c92ecaa None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:10.585326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1d7ba429-fbc8-414b-bfb0-834c1c92ecaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:10.603168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-443ff598-cde2-4771-a307-b1aa794d0e0e None None] MFA Rules not processed for user `90bad0adb01a4d34876826ac6fa18551`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:10.605088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-443ff598-cde2-4771-a307-b1aa794d0e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:10.704242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 902/1825] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:10.705660 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ff598-cde2-4771-a307-b1aa794d0e0e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:10.707227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-443ff598-cde2-4771-a307-b1aa794d0e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:10.731445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 924/1826] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:10.740214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ba429-fbc8-414b-bfb0-834c1c92ecaa None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:10.746103 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ba429-fbc8-414b-bfb0-834c1c92ecaa None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:10.751831 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041927f-4b76-423c-bfe5-e30bfb6647d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:10.752446 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041927f-4b76-423c-bfe5-e30bfb6647d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:10.752723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041927f-4b76-423c-bfe5-e30bfb6647d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:10.752995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041927f-4b76-423c-bfe5-e30bfb6647d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:10.753663 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041927f-4b76-423c-bfe5-e30bfb6647d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:38:10.753973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041927f-4b76-423c-bfe5-e30bfb6647d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:10.755606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041927f-4b76-423c-bfe5-e30bfb6647d8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:10.769890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 925/1827] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:38:10.774988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041927f-4b76-423c-bfe5-e30bfb6647d8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:10.785449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041927f-4b76-423c-bfe5-e30bfb6647d8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:10.791713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6f15d-caa6-4e37-8151-31aa7c9a4ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:10.802445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6f15d-caa6-4e37-8151-31aa7c9a4ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:10.802445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6f15d-caa6-4e37-8151-31aa7c9a4ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:10.802445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6f15d-caa6-4e37-8151-31aa7c9a4ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:10.802445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6f15d-caa6-4e37-8151-31aa7c9a4ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:10.802445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6f15d-caa6-4e37-8151-31aa7c9a4ebe admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:10.809831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ff598-cde2-4771-a307-b1aa794d0e0e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:10.814178 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f875804-eca8-4f6e-b4a5-b1536c9b01c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:10.814970 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f875804-eca8-4f6e-b4a5-b1536c9b01c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:10.815317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f875804-eca8-4f6e-b4a5-b1536c9b01c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:10.815598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f875804-eca8-4f6e-b4a5-b1536c9b01c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:10.828707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f875804-eca8-4f6e-b4a5-b1536c9b01c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:10.830742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f875804-eca8-4f6e-b4a5-b1536c9b01c7 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:10.862812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 903/1828] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:10.867177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f875804-eca8-4f6e-b4a5-b1536c9b01c7 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:10.870381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8f875804-eca8-4f6e-b4a5-b1536c9b01c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:10.894148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 926/1829] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:10 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:10.897481 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6f15d-caa6-4e37-8151-31aa7c9a4ebe admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:10.911938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f875804-eca8-4f6e-b4a5-b1536c9b01c7 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:11.023142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6f15d-caa6-4e37-8151-31aa7c9a4ebe admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:11.033126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a8a71-b8b2-4711-9f1b-cfb4153f3533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:11.035730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a8a71-b8b2-4711-9f1b-cfb4153f3533 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:11.035730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a8a71-b8b2-4711-9f1b-cfb4153f3533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:11.035730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a8a71-b8b2-4711-9f1b-cfb4153f3533 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:11.035730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a8a71-b8b2-4711-9f1b-cfb4153f3533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:11.036005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:11.036629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:11.037042 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:11.037439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] PATH_INFO: `/v3/projects/9bd51ffb38e0416da76cbf1771b2be9f` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:11.039674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd51ffb38e0416da76cbf1771b2be9f)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:11.041280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a8a71-b8b2-4711-9f1b-cfb4153f3533 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:11.042285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:11.210732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 927/1830] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:10 2022] POST /identity/v3/users => generated 286 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:11.215409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a8a71-b8b2-4711-9f1b-cfb4153f3533 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:11.232154 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd51ffb38e0416da76cbf1771b2be9f', 'request_id': 'req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:11.233776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a8a71-b8b2-4711-9f1b-cfb4153f3533 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:11.239619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd51ffb38e0416da76cbf1771b2be9f', 'request_id': 'req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:11.313304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3e67f-f1bd-48c1-828a-25bd09c661f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:11.313926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3e67f-f1bd-48c1-828a-25bd09c661f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:11.314249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3e67f-f1bd-48c1-828a-25bd09c661f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:11.314521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3e67f-f1bd-48c1-828a-25bd09c661f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:11.315180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3e67f-f1bd-48c1-828a-25bd09c661f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:11.316967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3e67f-f1bd-48c1-828a-25bd09c661f4 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:11.331920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 928/1831] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:11.332950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3e67f-f1bd-48c1-828a-25bd09c661f4 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:11.343902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3e67f-f1bd-48c1-828a-25bd09c661f4 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:11.346745 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd51ffb38e0416da76cbf1771b2be9f', 'request_id': 'req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:11.358065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbde36-4362-4b52-a419-93bc219e21c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:11.358977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbde36-4362-4b52-a419-93bc219e21c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:11.358977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbde36-4362-4b52-a419-93bc219e21c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:11.359299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbde36-4362-4b52-a419-93bc219e21c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:11.359902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbde36-4362-4b52-a419-93bc219e21c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:11.361582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbde36-4362-4b52-a419-93bc219e21c2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:11.363983 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] The token cache is being invalidate because project 9bd51ffb38e0416da76cbf1771b2be9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:11.364366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:11.371742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:11.377285 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 929/1832] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:11.381932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 904/1833] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:10 2022] DELETE /identity/v3/projects/9bd51ffb38e0416da76cbf1771b2be9f => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:11.382046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbde36-4362-4b52-a419-93bc219e21c2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:11.382046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-efcbde36-4362-4b52-a419-93bc219e21c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:11.383498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:11.386171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:11.395042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbde36-4362-4b52-a419-93bc219e21c2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:11.400928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:11.401560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:11.401834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:11.402106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] PATH_INFO: `/v3/users/6541b13be2e144859906f7fb799c8486` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:11.403288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6541b13be2e144859906f7fb799c8486)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:11.403509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c4f9a-e88f-4f39-ade3-d6cf627bd6d2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:11.405373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:11.409994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef363ea6-1f5c-4e9e-ace1-0d0833732882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:11.410682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363ea6-1f5c-4e9e-ace1-0d0833732882 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:11.411286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363ea6-1f5c-4e9e-ace1-0d0833732882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:11.411575 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363ea6-1f5c-4e9e-ace1-0d0833732882 admin admin] PATH_INFO: `/v3/projects/6e09fe7df9dc49c5a9e864dd07e2f1f3/users/a45e2e465c0948b887f03d5f535b0e67/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:11.430862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef363ea6-1f5c-4e9e-ace1-0d0833732882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e09fe7df9dc49c5a9e864dd07e2f1f3, user_id=a45e2e465c0948b887f03d5f535b0e67, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:11.433592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef363ea6-1f5c-4e9e-ace1-0d0833732882 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:11.508546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 905/1834] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:11 2022] PUT /identity/v3/projects/6e09fe7df9dc49c5a9e864dd07e2f1f3/users/a45e2e465c0948b887f03d5f535b0e67/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:11.519628 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0511cc9-5d47-4efe-b6bd-de86fe326744 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:11.520107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0511cc9-5d47-4efe-b6bd-de86fe326744 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:11.520400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0511cc9-5d47-4efe-b6bd-de86fe326744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:11.553694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f0511cc9-5d47-4efe-b6bd-de86fe326744 None None] MFA Rules not processed for user `a45e2e465c0948b887f03d5f535b0e67`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:11.555005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f0511cc9-5d47-4efe-b6bd-de86fe326744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:11.591804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6541b13be2e144859906f7fb799c8486'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:11.624933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 906/1835] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:11.629442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0511cc9-5d47-4efe-b6bd-de86fe326744 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:11.689822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6541b13be2e144859906f7fb799c8486'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:11.692183 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0511cc9-5d47-4efe-b6bd-de86fe326744 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:11.698328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f06d75-4f2f-4a47-895c-60a304d81372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:11.698941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06d75-4f2f-4a47-895c-60a304d81372 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:11.699219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06d75-4f2f-4a47-895c-60a304d81372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:11.699587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06d75-4f2f-4a47-895c-60a304d81372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:11.700254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f06d75-4f2f-4a47-895c-60a304d81372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:11.702205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f06d75-4f2f-4a47-895c-60a304d81372 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:11.722331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6541b13be2e144859906f7fb799c8486'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:11.734074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6541b13be2e144859906f7fb799c8486'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:11.734074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6541b13be2e144859906f7fb799c8486'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:11.737996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 930/1836] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:11 2022] DELETE /identity/v3/users/6541b13be2e144859906f7fb799c8486 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:11.753318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:11.827506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 907/1837] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:11 2022] POST /identity/v3/projects => generated 371 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:11.829237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:11.831294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:11.899591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7aef2f-47ce-408a-8da0-8d98e0037633 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:12.058258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0b323-cbb1-40d8-96f9-bf61c5d380d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:12.059475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0b323-cbb1-40d8-96f9-bf61c5d380d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.060339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0b323-cbb1-40d8-96f9-bf61c5d380d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.060764 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0b323-cbb1-40d8-96f9-bf61c5d380d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.061740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0b323-cbb1-40d8-96f9-bf61c5d380d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:12.063696 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0b323-cbb1-40d8-96f9-bf61c5d380d6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:12.080672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:12.175972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 931/1838] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:11 2022] POST /identity/v3/users => generated 292 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:12.189293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f06d75-4f2f-4a47-895c-60a304d81372 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:12.206925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f06d75-4f2f-4a47-895c-60a304d81372 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:12.212695 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc9830-1d96-4b79-bda7-d732c041a625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:12.213360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9830-1d96-4b79-bda7-d732c041a625 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.215207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9830-1d96-4b79-bda7-d732c041a625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.215207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9830-1d96-4b79-bda7-d732c041a625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.215207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc9830-1d96-4b79-bda7-d732c041a625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:12.218055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc9830-1d96-4b79-bda7-d732c041a625 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:12.234057 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 908/1839] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:12.235205 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0b323-cbb1-40d8-96f9-bf61c5d380d6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:12.253267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc9830-1d96-4b79-bda7-d732c041a625 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:12.256179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-89bc9830-1d96-4b79-bda7-d732c041a625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:12.271251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0b323-cbb1-40d8-96f9-bf61c5d380d6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:12.274237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc9830-1d96-4b79-bda7-d732c041a625 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:12.279807 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:12.280923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.281212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.281487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] PATH_INFO: `/v3/projects/7d3bb6a60b4541f5829e93d5161c9a1a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.282669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d3bb6a60b4541f5829e93d5161c9a1a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:12.283494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef5b14-d13c-42de-80da-5510697e775f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:12.284292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef5b14-d13c-42de-80da-5510697e775f admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.284575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef5b14-d13c-42de-80da-5510697e775f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.286709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:12.286832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef5b14-d13c-42de-80da-5510697e775f admin admin] PATH_INFO: `/v3/projects/30ad198874514bea908f376c3ceb786d/users/b9bca34d62e04301a310278a69bbbbb3/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.320944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef5b14-d13c-42de-80da-5510697e775f admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ad198874514bea908f376c3ceb786d, user_id=b9bca34d62e04301a310278a69bbbbb3, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:12.333975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef5b14-d13c-42de-80da-5510697e775f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:12.437519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 932/1840] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:12 2022] PUT /identity/v3/projects/30ad198874514bea908f376c3ceb786d/users/b9bca34d62e04301a310278a69bbbbb3/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:12.441549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef5b14-d13c-42de-80da-5510697e775f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:12.448132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d3bb6a60b4541f5829e93d5161c9a1a', 'request_id': 'req-16f9a975-d187-4765-bbba-06ec33c12d03'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:12.457335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d3bb6a60b4541f5829e93d5161c9a1a', 'request_id': 'req-16f9a975-d187-4765-bbba-06ec33c12d03'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:12.515319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef5b14-d13c-42de-80da-5510697e775f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:12.541942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d3bb6a60b4541f5829e93d5161c9a1a', 'request_id': 'req-16f9a975-d187-4765-bbba-06ec33c12d03'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:12.557031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] The token cache is being invalidate because project 7d3bb6a60b4541f5829e93d5161c9a1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:12.557656 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:12.560261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-16f9a975-d187-4765-bbba-06ec33c12d03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:12.562618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 909/1841] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:12 2022] DELETE /identity/v3/projects/7d3bb6a60b4541f5829e93d5161c9a1a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:12.566964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368dc08-48aa-4a6d-81c3-fc92e68f4821 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.567407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368dc08-48aa-4a6d-81c3-fc92e68f4821 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.567779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368dc08-48aa-4a6d-81c3-fc92e68f4821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.592075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6368dc08-48aa-4a6d-81c3-fc92e68f4821 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:12.592327 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ce952-218b-4c05-9350-1557f1fa342a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:12.592947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ce952-218b-4c05-9350-1557f1fa342a admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.593385 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ce952-218b-4c05-9350-1557f1fa342a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.593493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6368dc08-48aa-4a6d-81c3-fc92e68f4821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:12.593653 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ce952-218b-4c05-9350-1557f1fa342a admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.594302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ce952-218b-4c05-9350-1557f1fa342a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:12.596077 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ce952-218b-4c05-9350-1557f1fa342a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:12.603449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 910/1842] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:12.611245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a56c3f-5d7c-44f2-b011-dbab4b733679 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.611773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 933/1843] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:38:12.614606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a56c3f-5d7c-44f2-b011-dbab4b733679 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.614976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a56c3f-5d7c-44f2-b011-dbab4b733679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.615839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ce952-218b-4c05-9350-1557f1fa342a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:12.618208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ad7ce952-218b-4c05-9350-1557f1fa342a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:12.638693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ce952-218b-4c05-9350-1557f1fa342a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:12.644620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3fe9b-d667-41cb-beec-0fb0b0f96187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:12.645296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3fe9b-d667-41cb-beec-0fb0b0f96187 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.645571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3fe9b-d667-41cb-beec-0fb0b0f96187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.645974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3fe9b-d667-41cb-beec-0fb0b0f96187 admin admin] PATH_INFO: `/v3/projects/30ad198874514bea908f376c3ceb786d/users/b9bca34d62e04301a310278a69bbbbb3/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.646190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f0a56c3f-5d7c-44f2-b011-dbab4b733679 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:12.647457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f0a56c3f-5d7c-44f2-b011-dbab4b733679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:12.647901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3fe9b-d667-41cb-beec-0fb0b0f96187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ad198874514bea908f376c3ceb786d, user_id=b9bca34d62e04301a310278a69bbbbb3, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:12.650808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3fe9b-d667-41cb-beec-0fb0b0f96187 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:12.656240 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 911/1844] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:12.657646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95d6e3-3d91-418f-af4c-410b6749e875 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.657933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95d6e3-3d91-418f-af4c-410b6749e875 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.658206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95d6e3-3d91-418f-af4c-410b6749e875 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.659592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 912/1845] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:38:12.667189 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95d6e3-3d91-418f-af4c-410b6749e875 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:12.670510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 934/1846] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:12 2022] PUT /identity/v3/projects/30ad198874514bea908f376c3ceb786d/users/b9bca34d62e04301a310278a69bbbbb3/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:12.671879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058eda47-81d1-496b-8839-ae4f8001b864 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.672555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058eda47-81d1-496b-8839-ae4f8001b864 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.672555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058eda47-81d1-496b-8839-ae4f8001b864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.700309 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-058eda47-81d1-496b-8839-ae4f8001b864 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:12.701743 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-058eda47-81d1-496b-8839-ae4f8001b864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:12.740643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95d6e3-3d91-418f-af4c-410b6749e875 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:12.747590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd4342-f82f-45c4-9207-bb9cebc10447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:12.748821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd4342-f82f-45c4-9207-bb9cebc10447 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.749117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd4342-f82f-45c4-9207-bb9cebc10447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.749393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd4342-f82f-45c4-9207-bb9cebc10447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.750205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd4342-f82f-45c4-9207-bb9cebc10447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:38:12.750515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd4342-f82f-45c4-9207-bb9cebc10447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:12.753090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd4342-f82f-45c4-9207-bb9cebc10447 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:12.768342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 913/1847] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:38:12.773194 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 935/1848] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:38:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:12.773407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e8bbf-1a49-4aad-8245-90566b6f42e3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.774472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e8bbf-1a49-4aad-8245-90566b6f42e3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.774846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e8bbf-1a49-4aad-8245-90566b6f42e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.774971 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058eda47-81d1-496b-8839-ae4f8001b864 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:12.786212 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058eda47-81d1-496b-8839-ae4f8001b864 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:12.798315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4211480-5f6c-449c-9f34-d1c05dfc7207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:12.799062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211480-5f6c-449c-9f34-d1c05dfc7207 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:12.799339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211480-5f6c-449c-9f34-d1c05dfc7207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:12.799626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211480-5f6c-449c-9f34-d1c05dfc7207 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:12.804900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4211480-5f6c-449c-9f34-d1c05dfc7207 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:12.804900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4211480-5f6c-449c-9f34-d1c05dfc7207 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:12.847481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-559e8bbf-1a49-4aad-8245-90566b6f42e3 None None] MFA Rules not processed for user `b9bca34d62e04301a310278a69bbbbb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:12.848948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-559e8bbf-1a49-4aad-8245-90566b6f42e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:12.885929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:12.939392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 936/1849] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:12 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:12.940433 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4211480-5f6c-449c-9f34-d1c05dfc7207 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:12.942283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f4211480-5f6c-449c-9f34-d1c05dfc7207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:12.984340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 914/1850] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:12.984988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e8bbf-1a49-4aad-8245-90566b6f42e3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:12.997081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e8bbf-1a49-4aad-8245-90566b6f42e3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:13.038501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:13.047288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4211480-5f6c-449c-9f34-d1c05dfc7207 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:13.054175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cfd18-15b0-40f4-90fd-2006152b53c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:13.055471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cfd18-15b0-40f4-90fd-2006152b53c9 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:13.055712 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cfd18-15b0-40f4-90fd-2006152b53c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:13.055987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cfd18-15b0-40f4-90fd-2006152b53c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:13.066649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cfd18-15b0-40f4-90fd-2006152b53c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:13.068335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cfd18-15b0-40f4-90fd-2006152b53c9 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:13.080989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26577922-3461-4440-9e97-4ff9fec01b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:13.081789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 937/1851] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:13.081904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26577922-3461-4440-9e97-4ff9fec01b1b admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:13.082093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26577922-3461-4440-9e97-4ff9fec01b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:13.082509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26577922-3461-4440-9e97-4ff9fec01b1b admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:13.083247 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26577922-3461-4440-9e97-4ff9fec01b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:13.085315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26577922-3461-4440-9e97-4ff9fec01b1b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:13.089741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cfd18-15b0-40f4-90fd-2006152b53c9 service nova] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:13.091418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5a2cfd18-15b0-40f4-90fd-2006152b53c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:13.186059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cfd18-15b0-40f4-90fd-2006152b53c9 service nova] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:13.189180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f0b26-90fc-4c41-9a04-e5d8076227c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:13.189874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f0b26-90fc-4c41-9a04-e5d8076227c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:13.190224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f0b26-90fc-4c41-9a04-e5d8076227c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:13.190569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f0b26-90fc-4c41-9a04-e5d8076227c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:13.211901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 915/1852] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:12 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:13.217437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26577922-3461-4440-9e97-4ff9fec01b1b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:13.229852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26577922-3461-4440-9e97-4ff9fec01b1b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:13.262829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b80b16-ae61-4367-a632-e21fa82064fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:13.266131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b80b16-ae61-4367-a632-e21fa82064fa admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:13.266675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b80b16-ae61-4367-a632-e21fa82064fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:13.266959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b80b16-ae61-4367-a632-e21fa82064fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:13.267638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b80b16-ae61-4367-a632-e21fa82064fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:13.273804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b80b16-ae61-4367-a632-e21fa82064fa admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:13.301968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 916/1853] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:13.306622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b80b16-ae61-4367-a632-e21fa82064fa admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:13.324736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b80b16-ae61-4367-a632-e21fa82064fa admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:13.329540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddf531-b6f7-4f4b-866d-709d03d71d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:13.330227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddf531-b6f7-4f4b-866d-709d03d71d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:13.330735 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddf531-b6f7-4f4b-866d-709d03d71d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:13.331325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddf531-b6f7-4f4b-866d-709d03d71d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:13.332093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddf531-b6f7-4f4b-866d-709d03d71d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:13.334618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddf531-b6f7-4f4b-866d-709d03d71d98 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:13.349233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 917/1854] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:13.353813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddf531-b6f7-4f4b-866d-709d03d71d98 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:13.358903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f0b26-90fc-4c41-9a04-e5d8076227c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:13.360185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f0b26-90fc-4c41-9a04-e5d8076227c1 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:13.371150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddf531-b6f7-4f4b-866d-709d03d71d98 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:13.391781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe459b-a04f-4af5-830d-ea2801e661a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:13.392714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe459b-a04f-4af5-830d-ea2801e661a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:13.392999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe459b-a04f-4af5-830d-ea2801e661a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:13.393274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe459b-a04f-4af5-830d-ea2801e661a5 admin admin] PATH_INFO: `/v3/projects/d7c3c0f254af44b49f90d690033a5716/users/d6077fad2efa4a469ed3984d5f648aaf/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:13.405636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 938/1855] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:13.416792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe459b-a04f-4af5-830d-ea2801e661a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c3c0f254af44b49f90d690033a5716, user_id=d6077fad2efa4a469ed3984d5f648aaf, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:13.419551 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe459b-a04f-4af5-830d-ea2801e661a5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:13.519986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 918/1856] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:13 2022] PUT /identity/v3/projects/d7c3c0f254af44b49f90d690033a5716/users/d6077fad2efa4a469ed3984d5f648aaf/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:13.530528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f34b27-a845-4a0b-8751-6ed79217be10 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:13.534617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f34b27-a845-4a0b-8751-6ed79217be10 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:13.534617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f34b27-a845-4a0b-8751-6ed79217be10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:13.583688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e6f34b27-a845-4a0b-8751-6ed79217be10 None None] MFA Rules not processed for user `d6077fad2efa4a469ed3984d5f648aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:13.586679 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e6f34b27-a845-4a0b-8751-6ed79217be10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:13.646082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:13.683994 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 939/1857] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:13.688765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe459b-a04f-4af5-830d-ea2801e661a5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:13.715669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:13.790618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe459b-a04f-4af5-830d-ea2801e661a5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:13.796025 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba731974-fb12-46e5-bc39-6c2f729c1c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:13.796795 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba731974-fb12-46e5-bc39-6c2f729c1c51 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:13.796997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba731974-fb12-46e5-bc39-6c2f729c1c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:13.804357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba731974-fb12-46e5-bc39-6c2f729c1c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:13.805920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba731974-fb12-46e5-bc39-6c2f729c1c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:13.808408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba731974-fb12-46e5-bc39-6c2f729c1c51 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:13.916793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 919/1858] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:13 2022] POST /identity/v3/projects => generated 387 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:13.922449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f34b27-a845-4a0b-8751-6ed79217be10 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:13.939956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f34b27-a845-4a0b-8751-6ed79217be10 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:14.008724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f44047-7335-48ea-a49b-68bcff60f7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:14.009635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f44047-7335-48ea-a49b-68bcff60f7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:14.010128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f44047-7335-48ea-a49b-68bcff60f7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:14.010580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f44047-7335-48ea-a49b-68bcff60f7b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:14.011494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f44047-7335-48ea-a49b-68bcff60f7b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:14.014167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f44047-7335-48ea-a49b-68bcff60f7b0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:14.108708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 940/1859] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:13 2022] POST /identity/v3/users => generated 300 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:14.113642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba731974-fb12-46e5-bc39-6c2f729c1c51 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:14.120333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:14.131483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba731974-fb12-46e5-bc39-6c2f729c1c51 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:14.142031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d9b6a-0050-479a-8e5d-ec207de95346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:14.142781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d9b6a-0050-479a-8e5d-ec207de95346 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:14.143137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d9b6a-0050-479a-8e5d-ec207de95346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:14.143503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d9b6a-0050-479a-8e5d-ec207de95346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:14.153185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d9b6a-0050-479a-8e5d-ec207de95346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:14.153185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d9b6a-0050-479a-8e5d-ec207de95346 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:14.162547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 920/1860] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:14.166251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f44047-7335-48ea-a49b-68bcff60f7b0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:14.176726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f44047-7335-48ea-a49b-68bcff60f7b0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:14.183784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2ba30-d440-45e8-966b-8cb417e11a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:14.184672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2ba30-d440-45e8-966b-8cb417e11a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:14.185058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2ba30-d440-45e8-966b-8cb417e11a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:14.185573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2ba30-d440-45e8-966b-8cb417e11a52 admin admin] PATH_INFO: `/v3/projects/c9750f09e4544caba51897c841c4bfb5/users/ca0363f0d27b45b38482e344a21cd42c/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:14.202727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2ba30-d440-45e8-966b-8cb417e11a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9750f09e4544caba51897c841c4bfb5, user_id=ca0363f0d27b45b38482e344a21cd42c, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:14.205545 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2ba30-d440-45e8-966b-8cb417e11a52 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:14.303022 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 941/1861] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:14 2022] PUT /identity/v3/projects/c9750f09e4544caba51897c841c4bfb5/users/ca0363f0d27b45b38482e344a21cd42c/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:14.308598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d9b6a-0050-479a-8e5d-ec207de95346 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:14.326031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d9b6a-0050-479a-8e5d-ec207de95346 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:14.403773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722d317-6a22-4ba3-9342-9edeaaeb08d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:14.404423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722d317-6a22-4ba3-9342-9edeaaeb08d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:14.404704 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722d317-6a22-4ba3-9342-9edeaaeb08d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:14.404975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722d317-6a22-4ba3-9342-9edeaaeb08d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:14.408698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722d317-6a22-4ba3-9342-9edeaaeb08d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:14.408698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722d317-6a22-4ba3-9342-9edeaaeb08d0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:14.417374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 921/1862] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:14.422653 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2ba30-d440-45e8-966b-8cb417e11a52 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:14.438493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2ba30-d440-45e8-966b-8cb417e11a52 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:14.450533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3203e-8d1e-4d82-8cda-11a36d1a5548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:14.451253 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3203e-8d1e-4d82-8cda-11a36d1a5548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:14.451531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3203e-8d1e-4d82-8cda-11a36d1a5548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:14.451810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3203e-8d1e-4d82-8cda-11a36d1a5548 admin admin] PATH_INFO: `/v3/projects/c9750f09e4544caba51897c841c4bfb5/users/ca0363f0d27b45b38482e344a21cd42c/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:14.453893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3203e-8d1e-4d82-8cda-11a36d1a5548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9750f09e4544caba51897c841c4bfb5, user_id=ca0363f0d27b45b38482e344a21cd42c, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:14.457424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3203e-8d1e-4d82-8cda-11a36d1a5548 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:14.471286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 942/1863] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:14 2022] PUT /identity/v3/projects/c9750f09e4544caba51897c841c4bfb5/users/ca0363f0d27b45b38482e344a21cd42c/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:14.483183 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861c2c-3f16-4fc5-b04e-9b57385df601 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:14.483475 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861c2c-3f16-4fc5-b04e-9b57385df601 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:14.483772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861c2c-3f16-4fc5-b04e-9b57385df601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:14.515528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-45861c2c-3f16-4fc5-b04e-9b57385df601 None None] MFA Rules not processed for user `ca0363f0d27b45b38482e344a21cd42c`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:14.516895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-45861c2c-3f16-4fc5-b04e-9b57385df601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:14.565636 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:14.607480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 922/1864] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:14.627159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3203e-8d1e-4d82-8cda-11a36d1a5548 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:14.633608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45861c2c-3f16-4fc5-b04e-9b57385df601 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:14.692123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45861c2c-3f16-4fc5-b04e-9b57385df601 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:14.696138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1263be-7948-400e-b0be-a9c0626db6bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:14.696749 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1263be-7948-400e-b0be-a9c0626db6bc service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:14.697034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1263be-7948-400e-b0be-a9c0626db6bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:14.700179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1263be-7948-400e-b0be-a9c0626db6bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:14.709370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1263be-7948-400e-b0be-a9c0626db6bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:14.710363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1263be-7948-400e-b0be-a9c0626db6bc service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:14.714211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3203e-8d1e-4d82-8cda-11a36d1a5548 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:14.719046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:14.719685 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:14.720045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:14.720345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] PATH_INFO: `/v3/users/a45e2e465c0948b887f03d5f535b0e67` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:14.721161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 923/1865] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:38:14.721577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a45e2e465c0948b887f03d5f535b0e67)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:14.723629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:14.893866 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:14.943593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a45e2e465c0948b887f03d5f535b0e67'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:15.024392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a45e2e465c0948b887f03d5f535b0e67'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:15.040439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a45e2e465c0948b887f03d5f535b0e67'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:15.051824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a45e2e465c0948b887f03d5f535b0e67'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:15.052688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a45e2e465c0948b887f03d5f535b0e67'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:15.065404 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb24e8-a311-4945-96e4-4f6ec545389c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:15.065692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb24e8-a311-4945-96e4-4f6ec545389c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:15.065970 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb24e8-a311-4945-96e4-4f6ec545389c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:15.076805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 943/1866] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:14 2022] DELETE /identity/v3/users/a45e2e465c0948b887f03d5f535b0e67 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:15.101915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-bdbb24e8-a311-4945-96e4-4f6ec545389c None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:15.101915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bdbb24e8-a311-4945-96e4-4f6ec545389c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:15.174648 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 924/1867] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:15.186941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:15.188622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:15.308336 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584cbcd-00c7-4d6d-9774-2a2ecad19936 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:15.314171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41583d-b36e-4810-bfd2-ccbe1f56b7e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:15.314797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41583d-b36e-4810-bfd2-ccbe1f56b7e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:15.315074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41583d-b36e-4810-bfd2-ccbe1f56b7e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:15.315350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41583d-b36e-4810-bfd2-ccbe1f56b7e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:15.326028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41583d-b36e-4810-bfd2-ccbe1f56b7e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:15.327518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41583d-b36e-4810-bfd2-ccbe1f56b7e9 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:15.337183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 944/1868] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:15.479017 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:15.707221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:15.737611 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb24e8-a311-4945-96e4-4f6ec545389c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:15.739138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bdbb24e8-a311-4945-96e4-4f6ec545389c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:15.752523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb24e8-a311-4945-96e4-4f6ec545389c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:15.768307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:15.769310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:15.769310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:15.769738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] PATH_INFO: `/v3/projects/6e09fe7df9dc49c5a9e864dd07e2f1f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:15.771088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e09fe7df9dc49c5a9e864dd07e2f1f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:15.773032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:15.886309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e09fe7df9dc49c5a9e864dd07e2f1f3', 'request_id': 'req-056519e6-58b0-4c5a-857d-3c7acafa7472'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:15.892646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e09fe7df9dc49c5a9e864dd07e2f1f3', 'request_id': 'req-056519e6-58b0-4c5a-857d-3c7acafa7472'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:15.966986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e09fe7df9dc49c5a9e864dd07e2f1f3', 'request_id': 'req-056519e6-58b0-4c5a-857d-3c7acafa7472'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:15.977733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] The token cache is being invalidate because project 6e09fe7df9dc49c5a9e864dd07e2f1f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:15.978024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:15.978731 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:15.980343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 925/1869] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:15 2022] DELETE /identity/v3/projects/6e09fe7df9dc49c5a9e864dd07e2f1f3 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:15.984530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41583d-b36e-4810-bfd2-ccbe1f56b7e9 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:15.995342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41583d-b36e-4810-bfd2-ccbe1f56b7e9 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:16.010966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:16.012733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:16.054093 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d49a6-da83-44a2-a8a4-cfec0cf35ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:16.055576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d49a6-da83-44a2-a8a4-cfec0cf35ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:16.056103 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d49a6-da83-44a2-a8a4-cfec0cf35ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:16.056528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d49a6-da83-44a2-a8a4-cfec0cf35ff1 admin admin] PATH_INFO: `/v3/users/b9bca34d62e04301a310278a69bbbbb3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:16.057791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d49a6-da83-44a2-a8a4-cfec0cf35ff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9bca34d62e04301a310278a69bbbbb3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:16.059811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d49a6-da83-44a2-a8a4-cfec0cf35ff1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:16.089087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:16.093971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056519e6-58b0-4c5a-857d-3c7acafa7472 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:16.098689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:16.099480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:16.099861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:16.100198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] PATH_INFO: `/v3/users/d6077fad2efa4a469ed3984d5f648aaf` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:16.101360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6077fad2efa4a469ed3984d5f648aaf)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:16.103311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:16.218622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e67d49a6-da83-44a2-a8a4-cfec0cf35ff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9bca34d62e04301a310278a69bbbbb3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:16.279010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e67d49a6-da83-44a2-a8a4-cfec0cf35ff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9bca34d62e04301a310278a69bbbbb3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:16.281092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6077fad2efa4a469ed3984d5f648aaf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:16.293962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e67d49a6-da83-44a2-a8a4-cfec0cf35ff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9bca34d62e04301a310278a69bbbbb3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:16.300723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e67d49a6-da83-44a2-a8a4-cfec0cf35ff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9bca34d62e04301a310278a69bbbbb3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:16.301515 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e67d49a6-da83-44a2-a8a4-cfec0cf35ff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9bca34d62e04301a310278a69bbbbb3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:16.308459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 945/1870] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:15 2022] DELETE /identity/v3/users/b9bca34d62e04301a310278a69bbbbb3 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:16.354522 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6077fad2efa4a469ed3984d5f648aaf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:16.368184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6077fad2efa4a469ed3984d5f648aaf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:16.370322 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6077fad2efa4a469ed3984d5f648aaf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:16.371075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6077fad2efa4a469ed3984d5f648aaf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:16.387163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 926/1871] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:16 2022] DELETE /identity/v3/users/d6077fad2efa4a469ed3984d5f648aaf => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:16.391827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bea46-79b0-4bfd-8c3c-1b1b69e983d6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:16.392300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bea46-79b0-4bfd-8c3c-1b1b69e983d6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:16.392652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bea46-79b0-4bfd-8c3c-1b1b69e983d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:16.413122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-527bea46-79b0-4bfd-8c3c-1b1b69e983d6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:16.414381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-527bea46-79b0-4bfd-8c3c-1b1b69e983d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:16.477290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 946/1872] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:16.486691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:16.488135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:16.550301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc2e6a-079f-4b3c-b65a-f5be5cbf5295 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:16.558334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed83bb-d11b-4532-b5aa-03ee04900eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:16.558334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed83bb-d11b-4532-b5aa-03ee04900eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:16.558334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed83bb-d11b-4532-b5aa-03ee04900eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:16.558334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed83bb-d11b-4532-b5aa-03ee04900eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:16.568079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed83bb-d11b-4532-b5aa-03ee04900eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:16.569513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed83bb-d11b-4532-b5aa-03ee04900eb9 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:16.579935 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 927/1873] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:16.828947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bea46-79b0-4bfd-8c3c-1b1b69e983d6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:16.830411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-527bea46-79b0-4bfd-8c3c-1b1b69e983d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:16.842421 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bea46-79b0-4bfd-8c3c-1b1b69e983d6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:16.847285 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:16.847917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:16.851817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:16.851817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] PATH_INFO: `/v3/projects/30ad198874514bea908f376c3ceb786d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:16.851817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30ad198874514bea908f376c3ceb786d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:16.852578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:16.858923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:16.982652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30ad198874514bea908f376c3ceb786d', 'request_id': 'req-c827036d-ad27-4489-95bb-bf70addcc602'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:16.991060 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30ad198874514bea908f376c3ceb786d', 'request_id': 'req-c827036d-ad27-4489-95bb-bf70addcc602'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.067458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30ad198874514bea908f376c3ceb786d', 'request_id': 'req-c827036d-ad27-4489-95bb-bf70addcc602'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.079149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] The token cache is being invalidate because project 30ad198874514bea908f376c3ceb786d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:17.079440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.080134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.081740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 947/1874] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:16 2022] DELETE /identity/v3/projects/30ad198874514bea908f376c3ceb786d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:17.086563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ea66a-bfac-49ef-a7a9-ff05138622b3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:17.087047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ea66a-bfac-49ef-a7a9-ff05138622b3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:17.087132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ea66a-bfac-49ef-a7a9-ff05138622b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:17.095877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:17.109487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-fb8ea66a-bfac-49ef-a7a9-ff05138622b3 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:17.111847 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-fb8ea66a-bfac-49ef-a7a9-ff05138622b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:17.113003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:17.114576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:17.179201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 928/1875] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:17.186938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36efc5b-9a56-4a5c-952d-cf7ab663f46d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:17.191539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36efc5b-9a56-4a5c-952d-cf7ab663f46d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:17.191539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36efc5b-9a56-4a5c-952d-cf7ab663f46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:17.200140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827036d-ad27-4489-95bb-bf70addcc602 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:17.204883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:17.205526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:17.205800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:17.206082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] PATH_INFO: `/v3/projects/d7c3c0f254af44b49f90d690033a5716` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:17.207231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7c3c0f254af44b49f90d690033a5716)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:17.209202 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:17.212101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d36efc5b-9a56-4a5c-952d-cf7ab663f46d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:17.213378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d36efc5b-9a56-4a5c-952d-cf7ab663f46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:17.221750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 929/1876] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:17.225919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36efc5b-9a56-4a5c-952d-cf7ab663f46d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:17.234926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36efc5b-9a56-4a5c-952d-cf7ab663f46d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:17.240335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e10a7-61d8-4906-a195-1b6d5de5cb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:17.240923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e10a7-61d8-4906-a195-1b6d5de5cb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:17.241286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e10a7-61d8-4906-a195-1b6d5de5cb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:17.241512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e10a7-61d8-4906-a195-1b6d5de5cb2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:17.242280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e10a7-61d8-4906-a195-1b6d5de5cb2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:38:17.242615 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e10a7-61d8-4906-a195-1b6d5de5cb2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:17.244339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e10a7-61d8-4906-a195-1b6d5de5cb2b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:17.255756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 930/1877] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:38:17.261064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e10a7-61d8-4906-a195-1b6d5de5cb2b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:17.270728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e10a7-61d8-4906-a195-1b6d5de5cb2b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:17.276439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcb819-316d-43e4-9fee-6ade46d85cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:17.277036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcb819-316d-43e4-9fee-6ade46d85cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:17.277331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcb819-316d-43e4-9fee-6ade46d85cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:17.277618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcb819-316d-43e4-9fee-6ade46d85cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:17.278242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afcb819-316d-43e4-9fee-6ade46d85cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:17.280301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afcb819-316d-43e4-9fee-6ade46d85cf1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:17.343981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7c3c0f254af44b49f90d690033a5716', 'request_id': 'req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.350688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7c3c0f254af44b49f90d690033a5716', 'request_id': 'req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.360682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 931/1878] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:17 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:17.364770 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcb819-316d-43e4-9fee-6ade46d85cf1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:17.438890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7c3c0f254af44b49f90d690033a5716', 'request_id': 'req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.446239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcb819-316d-43e4-9fee-6ade46d85cf1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:17.460040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59681b79-e903-423a-ae76-9a6175dbc855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:17.460040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59681b79-e903-423a-ae76-9a6175dbc855 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:17.460040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59681b79-e903-423a-ae76-9a6175dbc855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:17.460040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59681b79-e903-423a-ae76-9a6175dbc855 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:17.460435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] The token cache is being invalidate because project d7c3c0f254af44b49f90d690033a5716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:17.460531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59681b79-e903-423a-ae76-9a6175dbc855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:17.460602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.460602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.460738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59681b79-e903-423a-ae76-9a6175dbc855 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:17.460843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 948/1879] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:17 2022] DELETE /identity/v3/projects/d7c3c0f254af44b49f90d690033a5716 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:17.464411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:17.466125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:17.498787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:17.545175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e66f8-6c4e-4b77-803c-2159c4f2e2bd admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:17.550088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:17.551148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:17.551429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:17.551728 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] PATH_INFO: `/v3/users/ca0363f0d27b45b38482e344a21cd42c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:17.553078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0363f0d27b45b38482e344a21cd42c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:17.555166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:17.600417 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 932/1880] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:17 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:17.603265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59681b79-e903-423a-ae76-9a6175dbc855 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:17.605055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-59681b79-e903-423a-ae76-9a6175dbc855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:17.618578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59681b79-e903-423a-ae76-9a6175dbc855 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:17.623488 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378d53f-cdf3-42aa-95e7-797b0ea6c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:17.624118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378d53f-cdf3-42aa-95e7-797b0ea6c080 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:17.624401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378d53f-cdf3-42aa-95e7-797b0ea6c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:17.624679 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378d53f-cdf3-42aa-95e7-797b0ea6c080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:17.625399 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378d53f-cdf3-42aa-95e7-797b0ea6c080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:17.627128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378d53f-cdf3-42aa-95e7-797b0ea6c080 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:17.642139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 933/1881] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:38:17.650204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378d53f-cdf3-42aa-95e7-797b0ea6c080 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:17.677129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378d53f-cdf3-42aa-95e7-797b0ea6c080 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:17.684899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edd7e1-7902-46c0-af54-98f78f851ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:17.685601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edd7e1-7902-46c0-af54-98f78f851ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:17.685971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edd7e1-7902-46c0-af54-98f78f851ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:17.687135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edd7e1-7902-46c0-af54-98f78f851ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:17.690465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edd7e1-7902-46c0-af54-98f78f851ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:17.692618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edd7e1-7902-46c0-af54-98f78f851ed5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:17.706850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 934/1882] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:38:17.710601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edd7e1-7902-46c0-af54-98f78f851ed5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:17.729689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edd7e1-7902-46c0-af54-98f78f851ed5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:17.736191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49049b-8513-410d-9a4a-b172f501ba40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:17.736927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49049b-8513-410d-9a4a-b172f501ba40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:17.737330 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49049b-8513-410d-9a4a-b172f501ba40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:17.737775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49049b-8513-410d-9a4a-b172f501ba40 admin admin] PATH_INFO: `/v3/projects/1a1a3b55ff2a419fa2a0507c05a9ec51/users/6f3d883d15d3429689e035d4b73b3a88/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:17.749462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0363f0d27b45b38482e344a21cd42c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.754125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49049b-8513-410d-9a4a-b172f501ba40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a1a3b55ff2a419fa2a0507c05a9ec51, user_id=6f3d883d15d3429689e035d4b73b3a88, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:17.757009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49049b-8513-410d-9a4a-b172f501ba40 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:17.850067 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0363f0d27b45b38482e344a21cd42c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.856500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 935/1883] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:17 2022] PUT /identity/v3/projects/1a1a3b55ff2a419fa2a0507c05a9ec51/users/6f3d883d15d3429689e035d4b73b3a88/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:17.866921 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ead09-460c-4cd4-b80c-e8f9622308c8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:17.868068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ead09-460c-4cd4-b80c-e8f9622308c8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:17.868353 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ead09-460c-4cd4-b80c-e8f9622308c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:17.869563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0363f0d27b45b38482e344a21cd42c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.875974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0363f0d27b45b38482e344a21cd42c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.877176 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0363f0d27b45b38482e344a21cd42c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:17.885235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:17.885864 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 949/1884] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:17 2022] DELETE /identity/v3/users/ca0363f0d27b45b38482e344a21cd42c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:17.904378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d48ead09-460c-4cd4-b80c-e8f9622308c8 None None] MFA Rules not processed for user `6f3d883d15d3429689e035d4b73b3a88`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:17.905624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d48ead09-460c-4cd4-b80c-e8f9622308c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:17.999975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 936/1885] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:18.005520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:18.008556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:18.078512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5caba-74d0-4f8a-ae8d-39fab6721da1 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:18.083780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9bab4-8a12-4743-990c-7fa41ed10f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:18.084618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9bab4-8a12-4743-990c-7fa41ed10f0f admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:18.084896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9bab4-8a12-4743-990c-7fa41ed10f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:18.085171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9bab4-8a12-4743-990c-7fa41ed10f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:18.086404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9bab4-8a12-4743-990c-7fa41ed10f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:18.089030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9bab4-8a12-4743-990c-7fa41ed10f0f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:18.230434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 950/1886] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:18 2022] POST /identity/v3/projects => generated 377 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:18.234707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ead09-460c-4cd4-b80c-e8f9622308c8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:18.246874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ead09-460c-4cd4-b80c-e8f9622308c8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:18.313531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a881e-4788-45eb-9226-f3afb12b2c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:18.314279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a881e-4788-45eb-9226-f3afb12b2c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:18.314557 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a881e-4788-45eb-9226-f3afb12b2c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:18.314827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a881e-4788-45eb-9226-f3afb12b2c14 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:18.315462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a881e-4788-45eb-9226-f3afb12b2c14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:18.317456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a881e-4788-45eb-9226-f3afb12b2c14 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:18.339087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9bab4-8a12-4743-990c-7fa41ed10f0f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:18.344299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5ef9bab4-8a12-4743-990c-7fa41ed10f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:18.360780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9bab4-8a12-4743-990c-7fa41ed10f0f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:18.365868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:18.366675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:18.367206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:18.367800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] PATH_INFO: `/v3/projects/c9750f09e4544caba51897c841c4bfb5` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:18.369163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9750f09e4544caba51897c841c4bfb5)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:18.371268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:18.402882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:18.491161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 937/1887] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:18 2022] POST /identity/v3/users => generated 295 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:18.498709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a881e-4788-45eb-9226-f3afb12b2c14 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:18.518408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a881e-4788-45eb-9226-f3afb12b2c14 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:18.528638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c077876-acce-40b6-bd43-764eb75490fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:18.530139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c077876-acce-40b6-bd43-764eb75490fd admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:18.530651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c077876-acce-40b6-bd43-764eb75490fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:18.531164 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c077876-acce-40b6-bd43-764eb75490fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:18.532432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c077876-acce-40b6-bd43-764eb75490fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:18.535668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c077876-acce-40b6-bd43-764eb75490fd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:18.546289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9750f09e4544caba51897c841c4bfb5', 'request_id': 'req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:18.556378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 938/1888] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:38:18.557946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9750f09e4544caba51897c841c4bfb5', 'request_id': 'req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:18.560291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c077876-acce-40b6-bd43-764eb75490fd admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:18.576674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c077876-acce-40b6-bd43-764eb75490fd admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:18.609856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9750f09e4544caba51897c841c4bfb5', 'request_id': 'req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:18.625819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] The token cache is being invalidate because project c9750f09e4544caba51897c841c4bfb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:18.626526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:18.627314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6588e8ce-b0a7-4cea-aaf6-2ec2ba74358e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:18.629925 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 951/1889] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:18 2022] DELETE /identity/v3/projects/c9750f09e4544caba51897c841c4bfb5 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:18.636213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71efeb5-ebde-408e-9e08-c5c875397509 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:18.636647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71efeb5-ebde-408e-9e08-c5c875397509 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:18.641839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71efeb5-ebde-408e-9e08-c5c875397509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:18.674001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c71efeb5-ebde-408e-9e08-c5c875397509 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:18.675628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c71efeb5-ebde-408e-9e08-c5c875397509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:18.699172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20952543-dad8-4f61-b6a2-d806da44c5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:18.704987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20952543-dad8-4f61-b6a2-d806da44c5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:18.704987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20952543-dad8-4f61-b6a2-d806da44c5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:18.704987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20952543-dad8-4f61-b6a2-d806da44c5fe admin admin] PATH_INFO: `/v3/projects/cd18b25144d74fd09d0769d86c2c64bf/users/1106ef2995cb4fa9a69a6c7135fcbc19/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:18.726891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20952543-dad8-4f61-b6a2-d806da44c5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd18b25144d74fd09d0769d86c2c64bf, user_id=1106ef2995cb4fa9a69a6c7135fcbc19, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:18.737629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20952543-dad8-4f61-b6a2-d806da44c5fe admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:18.764561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 952/1890] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:18.772888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30d247-6448-48df-86b1-4ade5809bb83 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:18.774258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30d247-6448-48df-86b1-4ade5809bb83 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:18.774540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30d247-6448-48df-86b1-4ade5809bb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:18.801543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-df30d247-6448-48df-86b1-4ade5809bb83 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:18.811960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-df30d247-6448-48df-86b1-4ade5809bb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:18.811960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 953/1891] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:18.817459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30d247-6448-48df-86b1-4ade5809bb83 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:18.823759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:18.832122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 939/1892] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:18 2022] PUT /identity/v3/projects/cd18b25144d74fd09d0769d86c2c64bf/users/1106ef2995cb4fa9a69a6c7135fcbc19/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:18.839518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20952543-dad8-4f61-b6a2-d806da44c5fe admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:18.841663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-20952543-dad8-4f61-b6a2-d806da44c5fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:18.850191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30d247-6448-48df-86b1-4ade5809bb83 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:18.964104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975a649-907b-427e-832e-c1134214335c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:18.967379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975a649-907b-427e-832e-c1134214335c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:18.967921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975a649-907b-427e-832e-c1134214335c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:18.972947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975a649-907b-427e-832e-c1134214335c admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:18.973525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975a649-907b-427e-832e-c1134214335c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:38:18.974208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975a649-907b-427e-832e-c1134214335c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:18.976081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975a649-907b-427e-832e-c1134214335c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:19.000500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 954/1893] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:38:19.004520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20952543-dad8-4f61-b6a2-d806da44c5fe admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:19.010888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8ac06-c81a-40e8-b8be-038552fd349f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:19.011518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8ac06-c81a-40e8-b8be-038552fd349f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.011803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8ac06-c81a-40e8-b8be-038552fd349f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.012117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8ac06-c81a-40e8-b8be-038552fd349f admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.012844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8ac06-c81a-40e8-b8be-038552fd349f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:19.014495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8ac06-c81a-40e8-b8be-038552fd349f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:19.015348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975a649-907b-427e-832e-c1134214335c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:19.030689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 940/1894] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:38:19.038254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8ac06-c81a-40e8-b8be-038552fd349f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:19.038559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975a649-907b-427e-832e-c1134214335c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:19.039618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d034e-68d0-4dc8-81c9-9d83942f838d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:19.040409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d034e-68d0-4dc8-81c9-9d83942f838d admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.040695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d034e-68d0-4dc8-81c9-9d83942f838d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.040969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d034e-68d0-4dc8-81c9-9d83942f838d admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.041599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d034e-68d0-4dc8-81c9-9d83942f838d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:19.043501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d034e-68d0-4dc8-81c9-9d83942f838d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:19.046460 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8ac06-c81a-40e8-b8be-038552fd349f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:19.051843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1079a-028e-4e81-a740-6caf0239e23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:19.052549 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1079a-028e-4e81-a740-6caf0239e23d admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.052825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1079a-028e-4e81-a740-6caf0239e23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.053115 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1079a-028e-4e81-a740-6caf0239e23d admin admin] PATH_INFO: `/v3/projects/cd18b25144d74fd09d0769d86c2c64bf/users/1106ef2995cb4fa9a69a6c7135fcbc19/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.056866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1079a-028e-4e81-a740-6caf0239e23d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd18b25144d74fd09d0769d86c2c64bf, user_id=1106ef2995cb4fa9a69a6c7135fcbc19, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:19.059969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1079a-028e-4e81-a740-6caf0239e23d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:19.081044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 941/1895] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:19 2022] PUT /identity/v3/projects/cd18b25144d74fd09d0769d86c2c64bf/users/1106ef2995cb4fa9a69a6c7135fcbc19/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:19.087736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1cbd-54ef-4a9e-9cee-21117e8f0215 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.088123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1cbd-54ef-4a9e-9cee-21117e8f0215 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.088405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1cbd-54ef-4a9e-9cee-21117e8f0215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.120890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-456c1cbd-54ef-4a9e-9cee-21117e8f0215 None None] MFA Rules not processed for user `1106ef2995cb4fa9a69a6c7135fcbc19`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:19.122175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-456c1cbd-54ef-4a9e-9cee-21117e8f0215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:19.158544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 955/1896] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:19 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:19.159649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d034e-68d0-4dc8-81c9-9d83942f838d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:19.180909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:19.198150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 942/1897] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:19.209593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c1cbd-54ef-4a9e-9cee-21117e8f0215 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:19.211998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-456c1cbd-54ef-4a9e-9cee-21117e8f0215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:19.232337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d034e-68d0-4dc8-81c9-9d83942f838d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:19.239849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78743e-cef7-42a0-ae37-045901478578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:19.240529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78743e-cef7-42a0-ae37-045901478578 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.240821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78743e-cef7-42a0-ae37-045901478578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.241125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78743e-cef7-42a0-ae37-045901478578 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.241729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78743e-cef7-42a0-ae37-045901478578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:19.244497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78743e-cef7-42a0-ae37-045901478578 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:19.282431 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c1cbd-54ef-4a9e-9cee-21117e8f0215 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:19.302945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:19.308806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 956/1898] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:19 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:19.310562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78743e-cef7-42a0-ae37-045901478578 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:19.323179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78743e-cef7-42a0-ae37-045901478578 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:19.338085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77407d85-0728-4c17-bc2e-cd17d305f776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:19.338701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59969c31-0823-47c5-988a-9605aa916b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:19.339086 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77407d85-0728-4c17-bc2e-cd17d305f776 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.339351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59969c31-0823-47c5-988a-9605aa916b43 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.339447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77407d85-0728-4c17-bc2e-cd17d305f776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.339696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59969c31-0823-47c5-988a-9605aa916b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.339781 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77407d85-0728-4c17-bc2e-cd17d305f776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.339860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59969c31-0823-47c5-988a-9605aa916b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.340442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77407d85-0728-4c17-bc2e-cd17d305f776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:19.342084 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-59969c31-0823-47c5-988a-9605aa916b43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:19.343554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77407d85-0728-4c17-bc2e-cd17d305f776 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:19.355761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 957/1899] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:19.359553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77407d85-0728-4c17-bc2e-cd17d305f776 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:19.375085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77407d85-0728-4c17-bc2e-cd17d305f776 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:19.380179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a352d-35b4-4026-88ea-73e8dd4176ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:19.381018 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a352d-35b4-4026-88ea-73e8dd4176ff admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.381144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a352d-35b4-4026-88ea-73e8dd4176ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.381468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a352d-35b4-4026-88ea-73e8dd4176ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.382214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a352d-35b4-4026-88ea-73e8dd4176ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:19.385097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a352d-35b4-4026-88ea-73e8dd4176ff admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:19.396214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 958/1900] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:19.400612 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a352d-35b4-4026-88ea-73e8dd4176ff admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:19.410395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a352d-35b4-4026-88ea-73e8dd4176ff admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:19.416760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cc2e0-9df5-4fc4-884a-5e67830a269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:19.417419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cc2e0-9df5-4fc4-884a-5e67830a269a admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.418344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cc2e0-9df5-4fc4-884a-5e67830a269a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.418344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cc2e0-9df5-4fc4-884a-5e67830a269a admin admin] PATH_INFO: `/v3/projects/31ac8511c9624bca8c792f09bc783bc8/users/294e15bab7034b0ca58d225022a966b0/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.419638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59969c31-0823-47c5-988a-9605aa916b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:19.421149 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59969c31-0823-47c5-988a-9605aa916b43 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:19.431060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 943/1901] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:19.434377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cc2e0-9df5-4fc4-884a-5e67830a269a admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ac8511c9624bca8c792f09bc783bc8, user_id=294e15bab7034b0ca58d225022a966b0, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:19.437095 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cc2e0-9df5-4fc4-884a-5e67830a269a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:19.547949 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 959/1902] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:19 2022] PUT /identity/v3/projects/31ac8511c9624bca8c792f09bc783bc8/users/294e15bab7034b0ca58d225022a966b0/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:19.559096 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afde7-8b56-4954-a4e3-630db4be1a10 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.559471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afde7-8b56-4954-a4e3-630db4be1a10 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.559832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afde7-8b56-4954-a4e3-630db4be1a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.588726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-5e5afde7-8b56-4954-a4e3-630db4be1a10 None None] MFA Rules not processed for user `294e15bab7034b0ca58d225022a966b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:19.589893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5e5afde7-8b56-4954-a4e3-630db4be1a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:19.655589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 944/1903] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:19.660357 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cc2e0-9df5-4fc4-884a-5e67830a269a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:19.741309 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cc2e0-9df5-4fc4-884a-5e67830a269a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:19.748302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0adb6-1863-42e1-b20a-811cffca7f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:19.748921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0adb6-1863-42e1-b20a-811cffca7f8e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.749202 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0adb6-1863-42e1-b20a-811cffca7f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.749493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0adb6-1863-42e1-b20a-811cffca7f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.750527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0adb6-1863-42e1-b20a-811cffca7f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:19.753733 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0adb6-1863-42e1-b20a-811cffca7f8e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:19.850177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 960/1904] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:19 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:19.855296 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5afde7-8b56-4954-a4e3-630db4be1a10 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:19.870892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5afde7-8b56-4954-a4e3-630db4be1a10 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:19.941799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd87d5-dc3b-4220-a9d1-d20efd9ef58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:19.942527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd87d5-dc3b-4220-a9d1-d20efd9ef58e admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:19.942872 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd87d5-dc3b-4220-a9d1-d20efd9ef58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:19.943918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd87d5-dc3b-4220-a9d1-d20efd9ef58e admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:19.944691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd87d5-dc3b-4220-a9d1-d20efd9ef58e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:19.949768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd87d5-dc3b-4220-a9d1-d20efd9ef58e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:19.964696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:20.012302 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 945/1905] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:19 2022] POST /identity/v3/users => generated 295 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:20.017826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0adb6-1863-42e1-b20a-811cffca7f8e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:20.033458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0adb6-1863-42e1-b20a-811cffca7f8e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:20.039806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa818c-69ec-44f3-a0f9-3d146e71bc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:20.040647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa818c-69ec-44f3-a0f9-3d146e71bc76 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:20.041180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa818c-69ec-44f3-a0f9-3d146e71bc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:20.041457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa818c-69ec-44f3-a0f9-3d146e71bc76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:20.042123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa818c-69ec-44f3-a0f9-3d146e71bc76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:20.043788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa818c-69ec-44f3-a0f9-3d146e71bc76 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:20.056344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 961/1906] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:20.061423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd87d5-dc3b-4220-a9d1-d20efd9ef58e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:20.079853 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd87d5-dc3b-4220-a9d1-d20efd9ef58e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:20.085248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e72fe0-9a3b-42d1-a38b-dc2c0e3b5e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:20.086722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e72fe0-9a3b-42d1-a38b-dc2c0e3b5e25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:20.087081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e72fe0-9a3b-42d1-a38b-dc2c0e3b5e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:20.087426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e72fe0-9a3b-42d1-a38b-dc2c0e3b5e25 admin admin] PATH_INFO: `/v3/projects/0f93d885ba7d4d739eb47e09c607aaa3/users/41d8725075a449abafad68ead09f1089/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:20.107331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e72fe0-9a3b-42d1-a38b-dc2c0e3b5e25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f93d885ba7d4d739eb47e09c607aaa3, user_id=41d8725075a449abafad68ead09f1089, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:20.110415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e72fe0-9a3b-42d1-a38b-dc2c0e3b5e25 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:20.214732 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 946/1907] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:20 2022] PUT /identity/v3/projects/0f93d885ba7d4d739eb47e09c607aaa3/users/41d8725075a449abafad68ead09f1089/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:20.218734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa818c-69ec-44f3-a0f9-3d146e71bc76 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:20.232172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa818c-69ec-44f3-a0f9-3d146e71bc76 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:20.281576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:20.308614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1962541-3aec-4f2a-a898-821f76520b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:20.309226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1962541-3aec-4f2a-a898-821f76520b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:20.309505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1962541-3aec-4f2a-a898-821f76520b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:20.310193 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1962541-3aec-4f2a-a898-821f76520b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:20.310486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1962541-3aec-4f2a-a898-821f76520b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:20.312113 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1962541-3aec-4f2a-a898-821f76520b42 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:20.327265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 962/1908] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:20.329866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e72fe0-9a3b-42d1-a38b-dc2c0e3b5e25 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:20.356298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e72fe0-9a3b-42d1-a38b-dc2c0e3b5e25 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:20.362667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66a59e-86cf-4e5f-a976-c487ca81ecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:20.363347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a59e-86cf-4e5f-a976-c487ca81ecaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:20.363625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a59e-86cf-4e5f-a976-c487ca81ecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:20.363897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a59e-86cf-4e5f-a976-c487ca81ecaa admin admin] PATH_INFO: `/v3/projects/0f93d885ba7d4d739eb47e09c607aaa3/users/41d8725075a449abafad68ead09f1089/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:20.365994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66a59e-86cf-4e5f-a976-c487ca81ecaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f93d885ba7d4d739eb47e09c607aaa3, user_id=41d8725075a449abafad68ead09f1089, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:20.369711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66a59e-86cf-4e5f-a976-c487ca81ecaa admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:20.381098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 947/1909] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:20 2022] PUT /identity/v3/projects/0f93d885ba7d4d739eb47e09c607aaa3/users/41d8725075a449abafad68ead09f1089/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:20.397774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba812f-68fb-433a-ba96-b1f5fc0380bb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:20.398067 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba812f-68fb-433a-ba96-b1f5fc0380bb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:20.398605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba812f-68fb-433a-ba96-b1f5fc0380bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:20.427758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-72ba812f-68fb-433a-ba96-b1f5fc0380bb None None] MFA Rules not processed for user `41d8725075a449abafad68ead09f1089`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:20.429091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-72ba812f-68fb-433a-ba96-b1f5fc0380bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:20.503591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 963/1910] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:20.547251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66a59e-86cf-4e5f-a976-c487ca81ecaa admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:20.609666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66a59e-86cf-4e5f-a976-c487ca81ecaa admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:20.622703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bffba-152c-4bea-b79a-d8a0c69977a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:20.622703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bffba-152c-4bea-b79a-d8a0c69977a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:20.622703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bffba-152c-4bea-b79a-d8a0c69977a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:20.622703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bffba-152c-4bea-b79a-d8a0c69977a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:20.624253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3bffba-152c-4bea-b79a-d8a0c69977a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:20.625699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3bffba-152c-4bea-b79a-d8a0c69977a8 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:20.636686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 948/1911] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:38:21.702510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba812f-68fb-433a-ba96-b1f5fc0380bb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:21.775072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba812f-68fb-433a-ba96-b1f5fc0380bb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:21.779820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:21.780629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:21.781044 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:21.781412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] PATH_INFO: `/v3/users/294e15bab7034b0ca58d225022a966b0` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:21.782664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=294e15bab7034b0ca58d225022a966b0)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:21.784814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:22.006865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '294e15bab7034b0ca58d225022a966b0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:22.095207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '294e15bab7034b0ca58d225022a966b0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:22.100632 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:22.109581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '294e15bab7034b0ca58d225022a966b0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:22.116628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '294e15bab7034b0ca58d225022a966b0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:22.117881 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '294e15bab7034b0ca58d225022a966b0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:22.124066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 964/1912] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:21 2022] DELETE /identity/v3/users/294e15bab7034b0ca58d225022a966b0 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:22.128642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0889f3f-d33f-4e39-a763-0409938b989f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:22.129104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0889f3f-d33f-4e39-a763-0409938b989f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:22.129193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0889f3f-d33f-4e39-a763-0409938b989f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:22.150633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a0889f3f-d33f-4e39-a763-0409938b989f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:22.151854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a0889f3f-d33f-4e39-a763-0409938b989f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:22.226039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 949/1913] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:22.237587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:22.239048 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:22.302714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecb01e-3ee3-4634-8f47-498e9d355fdc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:22.307520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67960ca-b175-4896-9c6b-c333d3cb0de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:22.307961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67960ca-b175-4896-9c6b-c333d3cb0de1 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:22.308345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67960ca-b175-4896-9c6b-c333d3cb0de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:22.308630 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67960ca-b175-4896-9c6b-c333d3cb0de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:22.326090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67960ca-b175-4896-9c6b-c333d3cb0de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:22.328436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67960ca-b175-4896-9c6b-c333d3cb0de1 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:22.341945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 965/1914] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:38:22.782230 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0889f3f-d33f-4e39-a763-0409938b989f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:22.783763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a0889f3f-d33f-4e39-a763-0409938b989f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:22.796139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0889f3f-d33f-4e39-a763-0409938b989f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:22.802689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:22.803439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:22.807305 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:22.807305 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] PATH_INFO: `/v3/projects/31ac8511c9624bca8c792f09bc783bc8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:22.807305 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ac8511c9624bca8c792f09bc783bc8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:22.808192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:22.925395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ac8511c9624bca8c792f09bc783bc8', 'request_id': 'req-a00415c3-ad66-4b54-995b-025ac0b68f50'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:22.927506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67960ca-b175-4896-9c6b-c333d3cb0de1 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:22.929102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a67960ca-b175-4896-9c6b-c333d3cb0de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:22.932337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ac8511c9624bca8c792f09bc783bc8', 'request_id': 'req-a00415c3-ad66-4b54-995b-025ac0b68f50'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:22.935111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:22.943332 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67960ca-b175-4896-9c6b-c333d3cb0de1 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:23.002156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472d7ac-835b-4674-acfe-0328821133d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:23.002778 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472d7ac-835b-4674-acfe-0328821133d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:23.003058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472d7ac-835b-4674-acfe-0328821133d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:23.003334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472d7ac-835b-4674-acfe-0328821133d0 admin admin] PATH_INFO: `/v3/users/5fed0c0e628e4468ae88225445693d2c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:23.004506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472d7ac-835b-4674-acfe-0328821133d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fed0c0e628e4468ae88225445693d2c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:23.006436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472d7ac-835b-4674-acfe-0328821133d0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:23.043922 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ac8511c9624bca8c792f09bc783bc8', 'request_id': 'req-a00415c3-ad66-4b54-995b-025ac0b68f50'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.055500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] The token cache is being invalidate because project 31ac8511c9624bca8c792f09bc783bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:23.055920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.056740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.058473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 950/1915] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:22 2022] DELETE /identity/v3/projects/31ac8511c9624bca8c792f09bc783bc8 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:23.062065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:23.063477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:23.078078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00415c3-ad66-4b54-995b-025ac0b68f50 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:23.082780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:23.083389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:23.083668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:23.083940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] PATH_INFO: `/v3/users/41d8725075a449abafad68ead09f1089` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:23.085136 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d8725075a449abafad68ead09f1089)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:23.087742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:23.233947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1472d7ac-835b-4674-acfe-0328821133d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fed0c0e628e4468ae88225445693d2c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.317674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1472d7ac-835b-4674-acfe-0328821133d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fed0c0e628e4468ae88225445693d2c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.327124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d8725075a449abafad68ead09f1089'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.334587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1472d7ac-835b-4674-acfe-0328821133d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fed0c0e628e4468ae88225445693d2c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.343616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1472d7ac-835b-4674-acfe-0328821133d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fed0c0e628e4468ae88225445693d2c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.344414 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1472d7ac-835b-4674-acfe-0328821133d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fed0c0e628e4468ae88225445693d2c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.350821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 966/1916] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:22 2022] DELETE /identity/v3/users/5fed0c0e628e4468ae88225445693d2c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:23.351115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:23.355203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18828ed3-c19d-47e5-bea5-248a31ba22e9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:23.355490 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18828ed3-c19d-47e5-bea5-248a31ba22e9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:23.355764 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18828ed3-c19d-47e5-bea5-248a31ba22e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:23.375380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-18828ed3-c19d-47e5-bea5-248a31ba22e9 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:23.376687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-18828ed3-c19d-47e5-bea5-248a31ba22e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:23.386586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d8725075a449abafad68ead09f1089'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.401832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d8725075a449abafad68ead09f1089'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.402590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d8725075a449abafad68ead09f1089'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.403218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d8725075a449abafad68ead09f1089'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:23.419898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 951/1917] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:23 2022] DELETE /identity/v3/users/41d8725075a449abafad68ead09f1089 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:23.453879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 967/1918] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:23.462687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:23.464186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:23.531001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b158b-3b16-40eb-9f09-fdd45962da8d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:23.534660 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db710f18-0132-40f7-b334-18c10c75b494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:23.535263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db710f18-0132-40f7-b334-18c10c75b494 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:23.535565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db710f18-0132-40f7-b334-18c10c75b494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:23.535882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db710f18-0132-40f7-b334-18c10c75b494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:23.597500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db710f18-0132-40f7-b334-18c10c75b494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:23.598791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db710f18-0132-40f7-b334-18c10c75b494 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:23.610934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 952/1919] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:23.940627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18828ed3-c19d-47e5-bea5-248a31ba22e9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:23.942078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-18828ed3-c19d-47e5-bea5-248a31ba22e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:23.955480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18828ed3-c19d-47e5-bea5-248a31ba22e9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:23.960319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:23.961773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:23.962331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:23.962672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] PATH_INFO: `/v3/projects/0f93d885ba7d4d739eb47e09c607aaa3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:23.964361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f93d885ba7d4d739eb47e09c607aaa3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:23.966317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:24.093033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f93d885ba7d4d739eb47e09c607aaa3', 'request_id': 'req-488bb9f6-2bea-4be0-bb58-7a899280d770'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.101886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f93d885ba7d4d739eb47e09c607aaa3', 'request_id': 'req-488bb9f6-2bea-4be0-bb58-7a899280d770'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.181680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f93d885ba7d4d739eb47e09c607aaa3', 'request_id': 'req-488bb9f6-2bea-4be0-bb58-7a899280d770'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.188839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db710f18-0132-40f7-b334-18c10c75b494 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:24.190393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-db710f18-0132-40f7-b334-18c10c75b494 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:24.196271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] The token cache is being invalidate because project 0f93d885ba7d4d739eb47e09c607aaa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:24.196590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.197442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-488bb9f6-2bea-4be0-bb58-7a899280d770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.201011 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 968/1920] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:23 2022] DELETE /identity/v3/projects/0f93d885ba7d4d739eb47e09c607aaa3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:24.204654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db710f18-0132-40f7-b334-18c10c75b494 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:24.205566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cc942-ce76-467d-a78f-2e71b3230b9b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:24.209354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cc942-ce76-467d-a78f-2e71b3230b9b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:24.209354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cc942-ce76-467d-a78f-2e71b3230b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:24.217871 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:24.227746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ec1cc942-ce76-467d-a78f-2e71b3230b9b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:24.229044 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ec1cc942-ce76-467d-a78f-2e71b3230b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:24.273356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:24.274015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:24.274424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:24.274714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] PATH_INFO: `/v3/projects/c407485082104169b5ded0ca119f1577` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:24.275854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c407485082104169b5ded0ca119f1577)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:24.277819 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:24.290909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 969/1921] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:24.298551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401ced5-695a-4468-ae29-cd17fce26b7b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:24.298846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401ced5-695a-4468-ae29-cd17fce26b7b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:24.299125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401ced5-695a-4468-ae29-cd17fce26b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:24.318686 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c401ced5-695a-4468-ae29-cd17fce26b7b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:24.319889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c401ced5-695a-4468-ae29-cd17fce26b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:24.331166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 970/1922] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:24.335443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401ced5-695a-4468-ae29-cd17fce26b7b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:24.347015 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401ced5-695a-4468-ae29-cd17fce26b7b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:24.349164 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17496a31-dd19-4d4c-9ff7-7ec978873d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:24.349743 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496a31-dd19-4d4c-9ff7-7ec978873d08 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:24.350014 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496a31-dd19-4d4c-9ff7-7ec978873d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:24.350283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496a31-dd19-4d4c-9ff7-7ec978873d08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:24.350941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17496a31-dd19-4d4c-9ff7-7ec978873d08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:38:24.351388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17496a31-dd19-4d4c-9ff7-7ec978873d08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:24.352988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17496a31-dd19-4d4c-9ff7-7ec978873d08 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:24.364480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 971/1923] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:38:24.369302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17496a31-dd19-4d4c-9ff7-7ec978873d08 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:24.374318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:24.379632 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17496a31-dd19-4d4c-9ff7-7ec978873d08 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:24.383448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c407485082104169b5ded0ca119f1577', 'request_id': 'req-812ef39e-6c84-49f7-b706-a0cb68321e41'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.390738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c407485082104169b5ded0ca119f1577', 'request_id': 'req-812ef39e-6c84-49f7-b706-a0cb68321e41'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.437105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf8c96-a183-4e90-b931-9f4053f60ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:24.437701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf8c96-a183-4e90-b931-9f4053f60ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:24.437977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf8c96-a183-4e90-b931-9f4053f60ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:24.438248 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf8c96-a183-4e90-b931-9f4053f60ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:24.439176 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf8c96-a183-4e90-b931-9f4053f60ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:24.441417 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf8c96-a183-4e90-b931-9f4053f60ee1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:24.460746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c407485082104169b5ded0ca119f1577', 'request_id': 'req-812ef39e-6c84-49f7-b706-a0cb68321e41'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.471569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] The token cache is being invalidate because project c407485082104169b5ded0ca119f1577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:24.471856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.472506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.473983 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 953/1924] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:24 2022] DELETE /identity/v3/projects/c407485082104169b5ded0ca119f1577 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:24.478556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:24.479923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:24.492748 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ef39e-6c84-49f7-b706-a0cb68321e41 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:24.498249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e462731-027e-4e5c-b647-57a2be1c1099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:24.498249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e462731-027e-4e5c-b647-57a2be1c1099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:24.498451 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e462731-027e-4e5c-b647-57a2be1c1099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:24.498687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e462731-027e-4e5c-b647-57a2be1c1099 admin admin] PATH_INFO: `/v3/users/280ce449e404404f9f20be0636bacbf8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:24.500146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e462731-027e-4e5c-b647-57a2be1c1099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=280ce449e404404f9f20be0636bacbf8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:24.502063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e462731-027e-4e5c-b647-57a2be1c1099 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:24.526006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 972/1925] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:24 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:24.530757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf8c96-a183-4e90-b931-9f4053f60ee1 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:24.533048 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4bdf8c96-a183-4e90-b931-9f4053f60ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:24.597927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf8c96-a183-4e90-b931-9f4053f60ee1 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:24.603538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892de921-54d8-4dc1-9646-1cfb2bc76430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:24.604190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de921-54d8-4dc1-9646-1cfb2bc76430 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:24.604471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de921-54d8-4dc1-9646-1cfb2bc76430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:24.604747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de921-54d8-4dc1-9646-1cfb2bc76430 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:24.605370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892de921-54d8-4dc1-9646-1cfb2bc76430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:24.607304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892de921-54d8-4dc1-9646-1cfb2bc76430 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:24.718118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 973/1926] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:24 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:24.724115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892de921-54d8-4dc1-9646-1cfb2bc76430 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:24.734151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892de921-54d8-4dc1-9646-1cfb2bc76430 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:24.738834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e229dc9-7f0c-4d68-b0b4-c9161fe9674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:24.739652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e229dc9-7f0c-4d68-b0b4-c9161fe9674e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:24.739765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e229dc9-7f0c-4d68-b0b4-c9161fe9674e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:24.740051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e229dc9-7f0c-4d68-b0b4-c9161fe9674e admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:24.740927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e229dc9-7f0c-4d68-b0b4-c9161fe9674e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:24.745674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e229dc9-7f0c-4d68-b0b4-c9161fe9674e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:24.754174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 974/1927] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:38:24.754819 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8e462731-027e-4e5c-b647-57a2be1c1099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280ce449e404404f9f20be0636bacbf8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.758113 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e229dc9-7f0c-4d68-b0b4-c9161fe9674e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:24.767822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e229dc9-7f0c-4d68-b0b4-c9161fe9674e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:24.774337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78039ea5-8cfe-4ca6-a328-7424371d063c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:24.774337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78039ea5-8cfe-4ca6-a328-7424371d063c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:24.774337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78039ea5-8cfe-4ca6-a328-7424371d063c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:24.774337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78039ea5-8cfe-4ca6-a328-7424371d063c admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:24.774706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78039ea5-8cfe-4ca6-a328-7424371d063c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:24.776292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78039ea5-8cfe-4ca6-a328-7424371d063c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:24.786009 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 975/1928] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:38:24.790109 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78039ea5-8cfe-4ca6-a328-7424371d063c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:24.799174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78039ea5-8cfe-4ca6-a328-7424371d063c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:24.803677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8e4c8-2233-4e77-952c-a181b0ced37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:24.805658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8e4c8-2233-4e77-952c-a181b0ced37a admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:24.805658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8e4c8-2233-4e77-952c-a181b0ced37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:24.805658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8e4c8-2233-4e77-952c-a181b0ced37a admin admin] PATH_INFO: `/v3/projects/567e2af75e1c4d1ab9e99ebb4c40b453/users/c95ed95b58f441cda0b0077ca6b2739d/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:24.818736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8e4c8-2233-4e77-952c-a181b0ced37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=567e2af75e1c4d1ab9e99ebb4c40b453, user_id=c95ed95b58f441cda0b0077ca6b2739d, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:24.821956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8e4c8-2233-4e77-952c-a181b0ced37a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:24.837652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8e462731-027e-4e5c-b647-57a2be1c1099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280ce449e404404f9f20be0636bacbf8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.850273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8e462731-027e-4e5c-b647-57a2be1c1099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280ce449e404404f9f20be0636bacbf8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.850868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8e462731-027e-4e5c-b647-57a2be1c1099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280ce449e404404f9f20be0636bacbf8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.851618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8e462731-027e-4e5c-b647-57a2be1c1099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280ce449e404404f9f20be0636bacbf8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:24.864340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 954/1929] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:24 2022] DELETE /identity/v3/users/280ce449e404404f9f20be0636bacbf8 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:24.924912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 976/1930] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:24 2022] PUT /identity/v3/projects/567e2af75e1c4d1ab9e99ebb4c40b453/users/c95ed95b58f441cda0b0077ca6b2739d/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:24.936964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9c5c3-d7d5-4739-8023-d86e9918a48a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:24.937258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9c5c3-d7d5-4739-8023-d86e9918a48a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:24.937539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9c5c3-d7d5-4739-8023-d86e9918a48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:24.962310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:24.971400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-cba9c5c3-d7d5-4739-8023-d86e9918a48a None None] MFA Rules not processed for user `c95ed95b58f441cda0b0077ca6b2739d`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:24.979029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cba9c5c3-d7d5-4739-8023-d86e9918a48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:25.041683 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 955/1931] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:25.046594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8e4c8-2233-4e77-952c-a181b0ced37a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:25.048754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dad8e4c8-2233-4e77-952c-a181b0ced37a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:25.088606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:25.124639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8e4c8-2233-4e77-952c-a181b0ced37a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:25.129717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff152dec-6d93-4e60-8d41-611bb5664877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:25.130353 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff152dec-6d93-4e60-8d41-611bb5664877 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.130805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff152dec-6d93-4e60-8d41-611bb5664877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.130907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff152dec-6d93-4e60-8d41-611bb5664877 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.131480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff152dec-6d93-4e60-8d41-611bb5664877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:25.134430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff152dec-6d93-4e60-8d41-611bb5664877 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:25.246428 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 977/1932] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:25 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:25.257075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9c5c3-d7d5-4739-8023-d86e9918a48a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:25.269007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9c5c3-d7d5-4739-8023-d86e9918a48a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:25.332391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645db05e-47d2-4311-b4d3-864069e2e394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:25.333166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645db05e-47d2-4311-b4d3-864069e2e394 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.333555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645db05e-47d2-4311-b4d3-864069e2e394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.333963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645db05e-47d2-4311-b4d3-864069e2e394 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.334729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645db05e-47d2-4311-b4d3-864069e2e394 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:25.337128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645db05e-47d2-4311-b4d3-864069e2e394 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:25.365471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff152dec-6d93-4e60-8d41-611bb5664877 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:25.367996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ff152dec-6d93-4e60-8d41-611bb5664877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:25.381030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff152dec-6d93-4e60-8d41-611bb5664877 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:25.392345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:25.392345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.392345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.392345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] PATH_INFO: `/v3/projects/d4a459882af2438a8c9875e22905b605` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.392822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4a459882af2438a8c9875e22905b605)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:25.392822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:25.416887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 956/1933] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:25 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:25.421752 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645db05e-47d2-4311-b4d3-864069e2e394 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:25.432796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645db05e-47d2-4311-b4d3-864069e2e394 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:25.437663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3771c-efe3-4ccd-8992-27005fc4cdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:25.438353 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3771c-efe3-4ccd-8992-27005fc4cdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.438689 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3771c-efe3-4ccd-8992-27005fc4cdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.439022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3771c-efe3-4ccd-8992-27005fc4cdc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.440958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3771c-efe3-4ccd-8992-27005fc4cdc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:25.441461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3771c-efe3-4ccd-8992-27005fc4cdc3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:25.452123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 957/1934] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:25.456380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3771c-efe3-4ccd-8992-27005fc4cdc3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:25.471479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3771c-efe3-4ccd-8992-27005fc4cdc3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:25.471706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8887b29-99aa-4de0-a4e5-d9e520a83f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:25.472411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8887b29-99aa-4de0-a4e5-d9e520a83f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.472680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8887b29-99aa-4de0-a4e5-d9e520a83f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.472957 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8887b29-99aa-4de0-a4e5-d9e520a83f43 admin admin] PATH_INFO: `/v3/projects/ed2466e7de2140e0aebb35ade8a760dd/users/11022171a9354e848ff2603eaae4d18b/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.488008 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8887b29-99aa-4de0-a4e5-d9e520a83f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2466e7de2140e0aebb35ade8a760dd, user_id=11022171a9354e848ff2603eaae4d18b, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:25.490699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8887b29-99aa-4de0-a4e5-d9e520a83f43 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:25.507540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4a459882af2438a8c9875e22905b605', 'request_id': 'req-69a1521b-eda8-4819-b995-0c400cd62fc8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:25.513976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4a459882af2438a8c9875e22905b605', 'request_id': 'req-69a1521b-eda8-4819-b995-0c400cd62fc8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:25.552416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 958/1935] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:25 2022] PUT /identity/v3/projects/ed2466e7de2140e0aebb35ade8a760dd/users/11022171a9354e848ff2603eaae4d18b/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:25.556902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8887b29-99aa-4de0-a4e5-d9e520a83f43 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:25.587440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4a459882af2438a8c9875e22905b605', 'request_id': 'req-69a1521b-eda8-4819-b995-0c400cd62fc8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:25.598973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] The token cache is being invalidate because project d4a459882af2438a8c9875e22905b605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:25.599381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:25.600155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-69a1521b-eda8-4819-b995-0c400cd62fc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:25.602495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 978/1936] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:25 2022] DELETE /identity/v3/projects/d4a459882af2438a8c9875e22905b605 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:25.607373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f2dd0-2f3c-4c2a-b95d-5d9685ee89ea None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.607702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f2dd0-2f3c-4c2a-b95d-5d9685ee89ea None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.608161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f2dd0-2f3c-4c2a-b95d-5d9685ee89ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.615359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:25.625699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8887b29-99aa-4de0-a4e5-d9e520a83f43 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:25.628540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b84f2dd0-2f3c-4c2a-b95d-5d9685ee89ea None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:25.629853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b84f2dd0-2f3c-4c2a-b95d-5d9685ee89ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:25.630379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3d22d-8ee4-4b0b-b78b-a14e84ba8f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:25.630979 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3d22d-8ee4-4b0b-b78b-a14e84ba8f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.632265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3d22d-8ee4-4b0b-b78b-a14e84ba8f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.632265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3d22d-8ee4-4b0b-b78b-a14e84ba8f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.632265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3d22d-8ee4-4b0b-b78b-a14e84ba8f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:25.634344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3d22d-8ee4-4b0b-b78b-a14e84ba8f3a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:25.638727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 979/1937] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:25.646547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddb157-62bd-49e1-8126-9c1e166c0e31 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.646859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddb157-62bd-49e1-8126-9c1e166c0e31 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.647166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddb157-62bd-49e1-8126-9c1e166c0e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.652542 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 959/1938] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:38:25.655187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3d22d-8ee4-4b0b-b78b-a14e84ba8f3a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:25.657012 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-87a3d22d-8ee4-4b0b-b78b-a14e84ba8f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:25.674730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3d22d-8ee4-4b0b-b78b-a14e84ba8f3a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:25.684876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5fddb157-62bd-49e1-8126-9c1e166c0e31 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:25.687001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5fddb157-62bd-49e1-8126-9c1e166c0e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:25.688899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e5eb8-d1f4-499b-8edf-4fa2381c8005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:25.689698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e5eb8-d1f4-499b-8edf-4fa2381c8005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.690083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e5eb8-d1f4-499b-8edf-4fa2381c8005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.690598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e5eb8-d1f4-499b-8edf-4fa2381c8005 admin admin] PATH_INFO: `/v3/projects/ed2466e7de2140e0aebb35ade8a760dd/users/11022171a9354e848ff2603eaae4d18b/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.692715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e5eb8-d1f4-499b-8edf-4fa2381c8005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2466e7de2140e0aebb35ade8a760dd, user_id=11022171a9354e848ff2603eaae4d18b, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:25.695520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e5eb8-d1f4-499b-8edf-4fa2381c8005 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:25.696423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 980/1939] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:25.700956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddb157-62bd-49e1-8126-9c1e166c0e31 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:25.717610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 960/1940] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:25 2022] PUT /identity/v3/projects/ed2466e7de2140e0aebb35ade8a760dd/users/11022171a9354e848ff2603eaae4d18b/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:25.717852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddb157-62bd-49e1-8126-9c1e166c0e31 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:25.721548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ba89b-8896-4171-80bc-9c96d3c0ba33 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.721887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ba89b-8896-4171-80bc-9c96d3c0ba33 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.722126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ba89b-8896-4171-80bc-9c96d3c0ba33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.772139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-218ba89b-8896-4171-80bc-9c96d3c0ba33 None None] MFA Rules not processed for user `11022171a9354e848ff2603eaae4d18b`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:25.774263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-218ba89b-8896-4171-80bc-9c96d3c0ba33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:25.802382 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5287cf-f4a8-43e8-854f-c887226f309f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:25.803172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5287cf-f4a8-43e8-854f-c887226f309f admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.803618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5287cf-f4a8-43e8-854f-c887226f309f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.804052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5287cf-f4a8-43e8-854f-c887226f309f admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.805767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5287cf-f4a8-43e8-854f-c887226f309f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:38:25.806324 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5287cf-f4a8-43e8-854f-c887226f309f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:25.808776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5287cf-f4a8-43e8-854f-c887226f309f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:25.829943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 981/1941] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:38:25.832314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5287cf-f4a8-43e8-854f-c887226f309f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:25.843711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5287cf-f4a8-43e8-854f-c887226f309f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:25.849719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68164902-828c-4e52-b144-dfa26806f8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:25.850490 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68164902-828c-4e52-b144-dfa26806f8fa admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:25.850871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68164902-828c-4e52-b144-dfa26806f8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:25.851313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68164902-828c-4e52-b144-dfa26806f8fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:25.852114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68164902-828c-4e52-b144-dfa26806f8fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:25.854153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68164902-828c-4e52-b144-dfa26806f8fa admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:25.945812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 961/1942] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:25.945812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ba89b-8896-4171-80bc-9c96d3c0ba33 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:25.945812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-218ba89b-8896-4171-80bc-9c96d3c0ba33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:25.951730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ba89b-8896-4171-80bc-9c96d3c0ba33 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:25.955449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233fa4e-f3f4-4f51-85ee-836ed215524d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:25.956094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233fa4e-f3f4-4f51-85ee-836ed215524d service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:26.006485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233fa4e-f3f4-4f51-85ee-836ed215524d service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:26.006485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233fa4e-f3f4-4f51-85ee-836ed215524d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:26.006485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233fa4e-f3f4-4f51-85ee-836ed215524d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:26.006485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233fa4e-f3f4-4f51-85ee-836ed215524d service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:26.006485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 962/1943] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:38:26.026522 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233fa4e-f3f4-4f51-85ee-836ed215524d service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:26.030635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a233fa4e-f3f4-4f51-85ee-836ed215524d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:26.043121 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233fa4e-f3f4-4f51-85ee-836ed215524d service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:26.046982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ae5e9-edc4-4947-92c3-886e7d16a185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:26.047727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ae5e9-edc4-4947-92c3-886e7d16a185 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:26.047902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ae5e9-edc4-4947-92c3-886e7d16a185 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:26.048209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ae5e9-edc4-4947-92c3-886e7d16a185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:26.050465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-176ae5e9-edc4-4947-92c3-886e7d16a185 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:26.117218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ae5e9-edc4-4947-92c3-886e7d16a185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:26.118652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ae5e9-edc4-4947-92c3-886e7d16a185 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:26.122728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:26.128763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 963/1944] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:38:26.250369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:26.250369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 982/1945] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:25 2022] POST /identity/v3/projects => generated 363 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:26.253442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ae5e9-edc4-4947-92c3-886e7d16a185 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:26.264588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ae5e9-edc4-4947-92c3-886e7d16a185 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:26.323009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bbf55-861a-49e2-84ac-69f8d829976f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:26.323621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bbf55-861a-49e2-84ac-69f8d829976f admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:26.323903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bbf55-861a-49e2-84ac-69f8d829976f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:26.324208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bbf55-861a-49e2-84ac-69f8d829976f admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:26.324943 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bbf55-861a-49e2-84ac-69f8d829976f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:26.327758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bbf55-861a-49e2-84ac-69f8d829976f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:26.377302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68164902-828c-4e52-b144-dfa26806f8fa admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:26.422376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 964/1946] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:26 2022] POST /identity/v3/users => generated 282 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:26.429349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bbf55-861a-49e2-84ac-69f8d829976f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:26.444538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bbf55-861a-49e2-84ac-69f8d829976f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:26.447950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68164902-828c-4e52-b144-dfa26806f8fa admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:26.453492 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057afbf-735f-419a-9173-5791d38f2692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:26.453492 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057afbf-735f-419a-9173-5791d38f2692 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:26.453492 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057afbf-735f-419a-9173-5791d38f2692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:26.453492 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057afbf-735f-419a-9173-5791d38f2692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:26.454066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01a488-06ec-48bc-a91d-7ddd958e3d6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:26.454066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01a488-06ec-48bc-a91d-7ddd958e3d6f service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:26.454066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01a488-06ec-48bc-a91d-7ddd958e3d6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:26.454315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057afbf-735f-419a-9173-5791d38f2692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:26.454391 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01a488-06ec-48bc-a91d-7ddd958e3d6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:26.455650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2d01a488-06ec-48bc-a91d-7ddd958e3d6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:26.455946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057afbf-735f-419a-9173-5791d38f2692 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:26.468409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 965/1947] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:38:26.472411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057afbf-735f-419a-9173-5791d38f2692 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:26.483741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057afbf-735f-419a-9173-5791d38f2692 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:26.488886 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4367b3-fab2-41fc-bea8-b16ce4ce415b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:26.489579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4367b3-fab2-41fc-bea8-b16ce4ce415b admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:26.490547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4367b3-fab2-41fc-bea8-b16ce4ce415b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:26.491053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4367b3-fab2-41fc-bea8-b16ce4ce415b admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:26.491612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4367b3-fab2-41fc-bea8-b16ce4ce415b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:26.494562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4367b3-fab2-41fc-bea8-b16ce4ce415b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:26.512662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 966/1948] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:26.515369 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4367b3-fab2-41fc-bea8-b16ce4ce415b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:26.526155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4367b3-fab2-41fc-bea8-b16ce4ce415b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:26.531556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a81baa-2f69-4c04-be0a-6e8fed164d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:26.532263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a81baa-2f69-4c04-be0a-6e8fed164d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:26.532542 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a81baa-2f69-4c04-be0a-6e8fed164d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:26.533077 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a81baa-2f69-4c04-be0a-6e8fed164d32 admin admin] PATH_INFO: `/v3/projects/3bd432faa5484aeb9a341f460c638bf9/users/4dee7384bb1144eab057ab14462c30ae/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:26.536896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01a488-06ec-48bc-a91d-7ddd958e3d6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:26.539745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01a488-06ec-48bc-a91d-7ddd958e3d6f service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:26.552191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a81baa-2f69-4c04-be0a-6e8fed164d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd432faa5484aeb9a341f460c638bf9, user_id=4dee7384bb1144eab057ab14462c30ae, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:26.558841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 983/1949] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:38:26.568615 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a81baa-2f69-4c04-be0a-6e8fed164d32 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:26.650691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 967/1950] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:26 2022] PUT /identity/v3/projects/3bd432faa5484aeb9a341f460c638bf9/users/4dee7384bb1144eab057ab14462c30ae/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:26.661580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93998d-0906-416d-9f0c-65f4db434c46 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:26.661868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93998d-0906-416d-9f0c-65f4db434c46 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:26.662153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93998d-0906-416d-9f0c-65f4db434c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:26.698244 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2f93998d-0906-416d-9f0c-65f4db434c46 None None] MFA Rules not processed for user `4dee7384bb1144eab057ab14462c30ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:26.699475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2f93998d-0906-416d-9f0c-65f4db434c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:26.773906 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 984/1951] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:26.779857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a81baa-2f69-4c04-be0a-6e8fed164d32 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:26.874724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a81baa-2f69-4c04-be0a-6e8fed164d32 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:26.879448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683e5d2-fbf4-4a34-af55-fd1ba4a18446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:26.880090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683e5d2-fbf4-4a34-af55-fd1ba4a18446 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:26.880374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683e5d2-fbf4-4a34-af55-fd1ba4a18446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:26.880648 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683e5d2-fbf4-4a34-af55-fd1ba4a18446 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:26.881310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683e5d2-fbf4-4a34-af55-fd1ba4a18446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:26.883806 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683e5d2-fbf4-4a34-af55-fd1ba4a18446 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:27.002632 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 968/1952] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:26 2022] POST /identity/v3/projects => generated 363 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:27.007056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93998d-0906-416d-9f0c-65f4db434c46 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:27.030769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93998d-0906-416d-9f0c-65f4db434c46 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:27.095062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:27.129806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8d034-3207-4755-842a-70c490303d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:27.131927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8d034-3207-4755-842a-70c490303d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:27.131927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8d034-3207-4755-842a-70c490303d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:27.131927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8d034-3207-4755-842a-70c490303d8b admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:27.131927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8d034-3207-4755-842a-70c490303d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:27.134162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8d034-3207-4755-842a-70c490303d8b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:27.219387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683e5d2-fbf4-4a34-af55-fd1ba4a18446 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:27.230581 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683e5d2-fbf4-4a34-af55-fd1ba4a18446 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:27.235427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985936f-0134-4563-a808-ee8849205cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:27.236125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985936f-0134-4563-a808-ee8849205cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:27.236471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985936f-0134-4563-a808-ee8849205cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:27.236750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985936f-0134-4563-a808-ee8849205cda admin admin] PATH_INFO: `/v3/users/c95ed95b58f441cda0b0077ca6b2739d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:27.240192 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 985/1953] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:27 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:27.241262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8d034-3207-4755-842a-70c490303d8b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:27.244180 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985936f-0134-4563-a808-ee8849205cda admin admin] RBAC: Authorizing `identity:delete_user(user_id=c95ed95b58f441cda0b0077ca6b2739d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:27.244180 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985936f-0134-4563-a808-ee8849205cda admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:27.255217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8d034-3207-4755-842a-70c490303d8b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:27.261449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc9de4-e36b-416a-8486-d1307dd9575b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:27.262080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc9de4-e36b-416a-8486-d1307dd9575b admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:27.262479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc9de4-e36b-416a-8486-d1307dd9575b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:27.262755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc9de4-e36b-416a-8486-d1307dd9575b admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:27.263399 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc9de4-e36b-416a-8486-d1307dd9575b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:27.265333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc9de4-e36b-416a-8486-d1307dd9575b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:27.271070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:27.277620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 986/1954] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:27.281749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc9de4-e36b-416a-8486-d1307dd9575b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:27.292070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc9de4-e36b-416a-8486-d1307dd9575b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:27.296871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d378-9d44-468e-accf-219b686b520b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:27.297518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d378-9d44-468e-accf-219b686b520b admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:27.299792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d378-9d44-468e-accf-219b686b520b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:27.299792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d378-9d44-468e-accf-219b686b520b admin admin] PATH_INFO: `/v3/projects/533d61ba28de40ccbc53168f370d790d/users/0a4e29fb91984a83b1af28581104e2a6/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:27.312851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d378-9d44-468e-accf-219b686b520b admin admin] RBAC: Authorizing `identity:create_grant(project_id=533d61ba28de40ccbc53168f370d790d, user_id=0a4e29fb91984a83b1af28581104e2a6, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:27.315502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d378-9d44-468e-accf-219b686b520b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:27.397616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 987/1955] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:27 2022] PUT /identity/v3/projects/533d61ba28de40ccbc53168f370d790d/users/0a4e29fb91984a83b1af28581104e2a6/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:27.399642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d378-9d44-468e-accf-219b686b520b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:27.440333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3985936f-0134-4563-a808-ee8849205cda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c95ed95b58f441cda0b0077ca6b2739d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:27.479587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d378-9d44-468e-accf-219b686b520b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:27.484579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09255eb6-fbc8-496e-b9b1-5cc6da9cde81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:27.485185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09255eb6-fbc8-496e-b9b1-5cc6da9cde81 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:27.485525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09255eb6-fbc8-496e-b9b1-5cc6da9cde81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:27.485758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09255eb6-fbc8-496e-b9b1-5cc6da9cde81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:27.486410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09255eb6-fbc8-496e-b9b1-5cc6da9cde81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:27.488078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09255eb6-fbc8-496e-b9b1-5cc6da9cde81 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:27.498320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 988/1956] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:27.502331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09255eb6-fbc8-496e-b9b1-5cc6da9cde81 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:27.513361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09255eb6-fbc8-496e-b9b1-5cc6da9cde81 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:27.517913 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853714e2-20d5-4eb4-b816-4a2bb870ee29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:27.518064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3985936f-0134-4563-a808-ee8849205cda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c95ed95b58f441cda0b0077ca6b2739d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:27.518559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853714e2-20d5-4eb4-b816-4a2bb870ee29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:27.518840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853714e2-20d5-4eb4-b816-4a2bb870ee29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:27.519116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853714e2-20d5-4eb4-b816-4a2bb870ee29 admin admin] PATH_INFO: `/v3/projects/533d61ba28de40ccbc53168f370d790d/users/0a4e29fb91984a83b1af28581104e2a6/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:27.521488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853714e2-20d5-4eb4-b816-4a2bb870ee29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=533d61ba28de40ccbc53168f370d790d, user_id=0a4e29fb91984a83b1af28581104e2a6, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:27.525664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853714e2-20d5-4eb4-b816-4a2bb870ee29 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:27.535511 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3985936f-0134-4563-a808-ee8849205cda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c95ed95b58f441cda0b0077ca6b2739d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:27.536396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3985936f-0134-4563-a808-ee8849205cda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c95ed95b58f441cda0b0077ca6b2739d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:27.546452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 989/1957] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:27 2022] PUT /identity/v3/projects/533d61ba28de40ccbc53168f370d790d/users/0a4e29fb91984a83b1af28581104e2a6/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:27.546662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3985936f-0134-4563-a808-ee8849205cda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c95ed95b58f441cda0b0077ca6b2739d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:27.548350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54289e94-a17c-4f14-bf5b-d021308acb7a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:27.548745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54289e94-a17c-4f14-bf5b-d021308acb7a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:27.549104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54289e94-a17c-4f14-bf5b-d021308acb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:27.550928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 969/1958] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:27 2022] DELETE /identity/v3/users/c95ed95b58f441cda0b0077ca6b2739d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:27.556174 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acab64-0029-4f5c-9768-69018c2ea84e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:27.559586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acab64-0029-4f5c-9768-69018c2ea84e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:27.559586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acab64-0029-4f5c-9768-69018c2ea84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:27.577630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-00acab64-0029-4f5c-9768-69018c2ea84e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:27.578868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-00acab64-0029-4f5c-9768-69018c2ea84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:27.584089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-54289e94-a17c-4f14-bf5b-d021308acb7a None None] MFA Rules not processed for user `0a4e29fb91984a83b1af28581104e2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:27.586398 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-54289e94-a17c-4f14-bf5b-d021308acb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:27.654791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 970/1959] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:27.665435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acab64-0029-4f5c-9768-69018c2ea84e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:27.666856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-00acab64-0029-4f5c-9768-69018c2ea84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:27.688666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 990/1960] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:27.706965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54289e94-a17c-4f14-bf5b-d021308acb7a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:27.708747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-54289e94-a17c-4f14-bf5b-d021308acb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:27.715729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:27.735356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acab64-0029-4f5c-9768-69018c2ea84e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:27.739553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194b34a-5ba5-4b22-8650-c690081bd7d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:27.740587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194b34a-5ba5-4b22-8650-c690081bd7d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:27.740587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194b34a-5ba5-4b22-8650-c690081bd7d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:27.740806 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194b34a-5ba5-4b22-8650-c690081bd7d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:27.750804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194b34a-5ba5-4b22-8650-c690081bd7d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:27.752272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194b34a-5ba5-4b22-8650-c690081bd7d2 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:27.763038 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 971/1961] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:27.772105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54289e94-a17c-4f14-bf5b-d021308acb7a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:27.776761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ada66e-7a63-4d1b-975c-fc8bda7f78c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:27.778284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ada66e-7a63-4d1b-975c-fc8bda7f78c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:27.778284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ada66e-7a63-4d1b-975c-fc8bda7f78c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:27.778284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ada66e-7a63-4d1b-975c-fc8bda7f78c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:27.788961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ada66e-7a63-4d1b-975c-fc8bda7f78c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:27.790589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ada66e-7a63-4d1b-975c-fc8bda7f78c5 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:27.801538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 991/1962] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:28.037200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194b34a-5ba5-4b22-8650-c690081bd7d2 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:28.039128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3194b34a-5ba5-4b22-8650-c690081bd7d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:28.056304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194b34a-5ba5-4b22-8650-c690081bd7d2 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:28.063421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013449de-0c30-4769-a4f5-c541cede446b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:28.064428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013449de-0c30-4769-a4f5-c541cede446b service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:28.064951 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013449de-0c30-4769-a4f5-c541cede446b service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:28.065445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013449de-0c30-4769-a4f5-c541cede446b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:28.067885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-013449de-0c30-4769-a4f5-c541cede446b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:28.125904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:28.149989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013449de-0c30-4769-a4f5-c541cede446b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:28.151577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013449de-0c30-4769-a4f5-c541cede446b service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:28.163773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 972/1963] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:28.259468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ada66e-7a63-4d1b-975c-fc8bda7f78c5 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:28.261595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-16ada66e-7a63-4d1b-975c-fc8bda7f78c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:28.277770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ada66e-7a63-4d1b-975c-fc8bda7f78c5 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:28.283081 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:28.284128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:28.284420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:28.284800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] PATH_INFO: `/v3/projects/567e2af75e1c4d1ab9e99ebb4c40b453` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:28.286119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=567e2af75e1c4d1ab9e99ebb4c40b453)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:28.288237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:28.437431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '567e2af75e1c4d1ab9e99ebb4c40b453', 'request_id': 'req-ea97374a-ab33-4930-89ac-ef29db6931e2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:28.445877 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '567e2af75e1c4d1ab9e99ebb4c40b453', 'request_id': 'req-ea97374a-ab33-4930-89ac-ef29db6931e2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:28.534870 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '567e2af75e1c4d1ab9e99ebb4c40b453', 'request_id': 'req-ea97374a-ab33-4930-89ac-ef29db6931e2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:28.547924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] The token cache is being invalidate because project 567e2af75e1c4d1ab9e99ebb4c40b453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:28.549496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:28.549496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:28.553177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 992/1964] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:28 2022] DELETE /identity/v3/projects/567e2af75e1c4d1ab9e99ebb4c40b453 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:28.557398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013449de-0c30-4769-a4f5-c541cede446b service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:28.573304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013449de-0c30-4769-a4f5-c541cede446b service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:28.689497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508dc525-38b7-4421-8ca9-64d46d558507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:28.689497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc525-38b7-4421-8ca9-64d46d558507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:28.689497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc525-38b7-4421-8ca9-64d46d558507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:28.690188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc525-38b7-4421-8ca9-64d46d558507 admin admin] PATH_INFO: `/v3/users/11022171a9354e848ff2603eaae4d18b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:28.691508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dc525-38b7-4421-8ca9-64d46d558507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11022171a9354e848ff2603eaae4d18b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:28.693805 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dc525-38b7-4421-8ca9-64d46d558507 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:28.888289 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-508dc525-38b7-4421-8ca9-64d46d558507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11022171a9354e848ff2603eaae4d18b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:28.963981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-508dc525-38b7-4421-8ca9-64d46d558507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11022171a9354e848ff2603eaae4d18b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:28.985722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-508dc525-38b7-4421-8ca9-64d46d558507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11022171a9354e848ff2603eaae4d18b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:28.986512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-508dc525-38b7-4421-8ca9-64d46d558507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11022171a9354e848ff2603eaae4d18b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:28.987123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-508dc525-38b7-4421-8ca9-64d46d558507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11022171a9354e848ff2603eaae4d18b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:29.000359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 973/1965] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:28 2022] DELETE /identity/v3/users/11022171a9354e848ff2603eaae4d18b => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:29.435285 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:29.436875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:29.457930 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97374a-ab33-4930-89ac-ef29db6931e2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:29.549036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:29.550453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:29.550757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:29.551121 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] PATH_INFO: `/v3/projects/ed2466e7de2140e0aebb35ade8a760dd` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:29.552774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2466e7de2140e0aebb35ade8a760dd)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:29.555632 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:29.689675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:29.704148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2466e7de2140e0aebb35ade8a760dd', 'request_id': 'req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:29.710218 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2466e7de2140e0aebb35ade8a760dd', 'request_id': 'req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:29.799420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2466e7de2140e0aebb35ade8a760dd', 'request_id': 'req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:29.812097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] The token cache is being invalidate because project ed2466e7de2140e0aebb35ade8a760dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:29.812400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:29.813054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1b7e9c33-3627-4ece-b0a0-3ecf70c72d4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:29.814926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 993/1966] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:29 2022] DELETE /identity/v3/projects/ed2466e7de2140e0aebb35ade8a760dd => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:29.821684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265e244-4391-4695-ba38-1067440ae1f8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:29.822213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265e244-4391-4695-ba38-1067440ae1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:29.822472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265e244-4391-4695-ba38-1067440ae1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:29.844064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b265e244-4391-4695-ba38-1067440ae1f8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:29.845333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b265e244-4391-4695-ba38-1067440ae1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:29.896543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:29.904785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 974/1967] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:29.912840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198ed07-5f98-4c91-94b5-c92a36f0a1c2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:29.913122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198ed07-5f98-4c91-94b5-c92a36f0a1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:29.913393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198ed07-5f98-4c91-94b5-c92a36f0a1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:29.936502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1198ed07-5f98-4c91-94b5-c92a36f0a1c2 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:29.937828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1198ed07-5f98-4c91-94b5-c92a36f0a1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:29.946259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 994/1968] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:29.950286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265e244-4391-4695-ba38-1067440ae1f8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:29.960229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265e244-4391-4695-ba38-1067440ae1f8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:29.964961 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7ff95-9ec3-4e1f-92c1-b5a86ba8845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:29.970364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7ff95-9ec3-4e1f-92c1-b5a86ba8845b admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:29.970364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7ff95-9ec3-4e1f-92c1-b5a86ba8845b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:29.970364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7ff95-9ec3-4e1f-92c1-b5a86ba8845b admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:29.970364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7ff95-9ec3-4e1f-92c1-b5a86ba8845b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:38:29.970364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7ff95-9ec3-4e1f-92c1-b5a86ba8845b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:29.970364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7ff95-9ec3-4e1f-92c1-b5a86ba8845b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:29.981219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 975/1969] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:38:29.985491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198ed07-5f98-4c91-94b5-c92a36f0a1c2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:29.995200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198ed07-5f98-4c91-94b5-c92a36f0a1c2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:30.000123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ef3e5-c33c-4171-8a89-14a058746035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:30.000715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ef3e5-c33c-4171-8a89-14a058746035 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:30.000991 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ef3e5-c33c-4171-8a89-14a058746035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:30.001263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ef3e5-c33c-4171-8a89-14a058746035 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:30.001891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ef3e5-c33c-4171-8a89-14a058746035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:30.004052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ef3e5-c33c-4171-8a89-14a058746035 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:30.095106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 995/1970] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:29 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:30.096183 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7ff95-9ec3-4e1f-92c1-b5a86ba8845b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:30.106920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7ff95-9ec3-4e1f-92c1-b5a86ba8845b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:30.165337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb62879-8c85-40c5-820b-60c68bc75a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:30.165945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb62879-8c85-40c5-820b-60c68bc75a3c admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:30.166282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb62879-8c85-40c5-820b-60c68bc75a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:30.166492 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb62879-8c85-40c5-820b-60c68bc75a3c admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:30.167120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb62879-8c85-40c5-820b-60c68bc75a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:30.169065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb62879-8c85-40c5-820b-60c68bc75a3c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:30.272411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 976/1971] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:30 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:30.278054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ef3e5-c33c-4171-8a89-14a058746035 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:30.290175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ef3e5-c33c-4171-8a89-14a058746035 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:30.294904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e614d7-b8f6-4be8-b67a-c3bb9b25bb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:30.295499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e614d7-b8f6-4be8-b67a-c3bb9b25bb3f admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:30.295774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e614d7-b8f6-4be8-b67a-c3bb9b25bb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:30.296078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e614d7-b8f6-4be8-b67a-c3bb9b25bb3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:30.296734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e614d7-b8f6-4be8-b67a-c3bb9b25bb3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:30.298305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e614d7-b8f6-4be8-b67a-c3bb9b25bb3f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:30.308879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 996/1972] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:30.312918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb62879-8c85-40c5-820b-60c68bc75a3c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:30.323706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb62879-8c85-40c5-820b-60c68bc75a3c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:30.328340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d57d4-50f6-40a2-b193-66cc54b7a7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:30.328933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d57d4-50f6-40a2-b193-66cc54b7a7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:30.329210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d57d4-50f6-40a2-b193-66cc54b7a7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:30.329602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d57d4-50f6-40a2-b193-66cc54b7a7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:30.330249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d57d4-50f6-40a2-b193-66cc54b7a7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:30.331814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d57d4-50f6-40a2-b193-66cc54b7a7d0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:30.344602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 977/1973] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:30.348612 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e614d7-b8f6-4be8-b67a-c3bb9b25bb3f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:30.364126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e614d7-b8f6-4be8-b67a-c3bb9b25bb3f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:30.369417 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220332f3-4769-4219-ac0e-5f526d1c71fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:30.370086 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220332f3-4769-4219-ac0e-5f526d1c71fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:30.370361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220332f3-4769-4219-ac0e-5f526d1c71fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:30.370636 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220332f3-4769-4219-ac0e-5f526d1c71fc admin admin] PATH_INFO: `/v3/projects/46be5db93569499fbe8169151eca4e1b/users/aaf81fb89c8342ed92216594913ca604/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:30.386475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220332f3-4769-4219-ac0e-5f526d1c71fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=46be5db93569499fbe8169151eca4e1b, user_id=aaf81fb89c8342ed92216594913ca604, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:30.389193 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220332f3-4769-4219-ac0e-5f526d1c71fc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:30.473737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 997/1974] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:30 2022] PUT /identity/v3/projects/46be5db93569499fbe8169151eca4e1b/users/aaf81fb89c8342ed92216594913ca604/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:30.485818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4fb99-0e13-4632-b81f-47e8af59ea27 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:30.486103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4fb99-0e13-4632-b81f-47e8af59ea27 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:30.486395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4fb99-0e13-4632-b81f-47e8af59ea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:30.517355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-50d4fb99-0e13-4632-b81f-47e8af59ea27 None None] MFA Rules not processed for user `aaf81fb89c8342ed92216594913ca604`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:30.518678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-50d4fb99-0e13-4632-b81f-47e8af59ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:30.588153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 978/1975] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:30.589941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220332f3-4769-4219-ac0e-5f526d1c71fc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:30.659620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220332f3-4769-4219-ac0e-5f526d1c71fc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:30.670225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1702dd-3386-4e5a-b25e-0f4e741a5abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:30.670897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1702dd-3386-4e5a-b25e-0f4e741a5abc admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:30.671209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1702dd-3386-4e5a-b25e-0f4e741a5abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:30.671482 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1702dd-3386-4e5a-b25e-0f4e741a5abc admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:30.672142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1702dd-3386-4e5a-b25e-0f4e741a5abc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:30.674467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1702dd-3386-4e5a-b25e-0f4e741a5abc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:30.765700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 998/1976] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:30 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:30.769606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4fb99-0e13-4632-b81f-47e8af59ea27 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:30.780560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4fb99-0e13-4632-b81f-47e8af59ea27 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:30.829518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:30.836800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77488a6c-f240-44df-a133-39a11a74d04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:30.841929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488a6c-f240-44df-a133-39a11a74d04b admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:30.841929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488a6c-f240-44df-a133-39a11a74d04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:30.841929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488a6c-f240-44df-a133-39a11a74d04b admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:30.841929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77488a6c-f240-44df-a133-39a11a74d04b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:30.841929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77488a6c-f240-44df-a133-39a11a74d04b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:30.918705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 979/1977] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:30 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:30.922620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1702dd-3386-4e5a-b25e-0f4e741a5abc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:30.932945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1702dd-3386-4e5a-b25e-0f4e741a5abc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:30.938325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041f2fc-9aac-4835-b6b8-a2de2e06312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:30.938985 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041f2fc-9aac-4835-b6b8-a2de2e06312a admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:30.939267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041f2fc-9aac-4835-b6b8-a2de2e06312a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:30.939544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041f2fc-9aac-4835-b6b8-a2de2e06312a admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:30.940227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041f2fc-9aac-4835-b6b8-a2de2e06312a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:30.941877 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041f2fc-9aac-4835-b6b8-a2de2e06312a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:30.952644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 999/1978] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:30.956816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77488a6c-f240-44df-a133-39a11a74d04b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:30.967426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77488a6c-f240-44df-a133-39a11a74d04b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:30.972221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5aa2d-15e1-4401-add9-1c0260cfaf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:30.972655 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5aa2d-15e1-4401-add9-1c0260cfaf43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:30.972931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5aa2d-15e1-4401-add9-1c0260cfaf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:30.973209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5aa2d-15e1-4401-add9-1c0260cfaf43 admin admin] PATH_INFO: `/v3/projects/65b8a1b404144673ade05d3245ab3b87/users/654d0760846846819d5d3dc8ee092ea4/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:30.989152 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5aa2d-15e1-4401-add9-1c0260cfaf43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b8a1b404144673ade05d3245ab3b87, user_id=654d0760846846819d5d3dc8ee092ea4, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:30.991860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5aa2d-15e1-4401-add9-1c0260cfaf43 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:31.066913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 980/1979] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:30 2022] PUT /identity/v3/projects/65b8a1b404144673ade05d3245ab3b87/users/654d0760846846819d5d3dc8ee092ea4/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:31.071002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041f2fc-9aac-4835-b6b8-a2de2e06312a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:31.080973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041f2fc-9aac-4835-b6b8-a2de2e06312a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:31.110181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:31.137506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf7984-6c42-400a-8551-a4f29d4cd601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:31.138177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7984-6c42-400a-8551-a4f29d4cd601 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:31.138638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7984-6c42-400a-8551-a4f29d4cd601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:31.139098 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7984-6c42-400a-8551-a4f29d4cd601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:31.140321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf7984-6c42-400a-8551-a4f29d4cd601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:31.142526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf7984-6c42-400a-8551-a4f29d4cd601 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:31.154034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1000/1980] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:31.158429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5aa2d-15e1-4401-add9-1c0260cfaf43 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:31.169817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5aa2d-15e1-4401-add9-1c0260cfaf43 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:31.174455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b263a-fd08-4781-8c87-5df76746a84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:31.175100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b263a-fd08-4781-8c87-5df76746a84d admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:31.175380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b263a-fd08-4781-8c87-5df76746a84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:31.179363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b263a-fd08-4781-8c87-5df76746a84d admin admin] PATH_INFO: `/v3/projects/65b8a1b404144673ade05d3245ab3b87/users/654d0760846846819d5d3dc8ee092ea4/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:31.179363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b263a-fd08-4781-8c87-5df76746a84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b8a1b404144673ade05d3245ab3b87, user_id=654d0760846846819d5d3dc8ee092ea4, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:31.180577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b263a-fd08-4781-8c87-5df76746a84d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:31.191451 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 981/1981] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:31 2022] PUT /identity/v3/projects/65b8a1b404144673ade05d3245ab3b87/users/654d0760846846819d5d3dc8ee092ea4/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:31.201940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1f745-167a-49d5-9845-ac3f0eef0a40 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:31.202320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1f745-167a-49d5-9845-ac3f0eef0a40 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:31.202606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1f745-167a-49d5-9845-ac3f0eef0a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:31.233322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9ee1f745-167a-49d5-9845-ac3f0eef0a40 None None] MFA Rules not processed for user `654d0760846846819d5d3dc8ee092ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:31.234539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9ee1f745-167a-49d5-9845-ac3f0eef0a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:31.301841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1001/1982] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:31.312972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b263a-fd08-4781-8c87-5df76746a84d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:31.314746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a44b263a-fd08-4781-8c87-5df76746a84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:31.373062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b263a-fd08-4781-8c87-5df76746a84d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:31.376704 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f842d-81cd-48ef-860c-cc91a01e7200 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:31.377303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f842d-81cd-48ef-860c-cc91a01e7200 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:31.377582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f842d-81cd-48ef-860c-cc91a01e7200 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:31.377856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f842d-81cd-48ef-860c-cc91a01e7200 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:31.433644 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f842d-81cd-48ef-860c-cc91a01e7200 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:31.434986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f842d-81cd-48ef-860c-cc91a01e7200 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:31.443760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 982/1983] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:41.298906 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=49 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:41.439744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:44.101975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1f745-167a-49d5-9845-ac3f0eef0a40 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:44.103457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9ee1f745-167a-49d5-9845-ac3f0eef0a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:44.115205 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1f745-167a-49d5-9845-ac3f0eef0a40 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:44.119210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc0ad0-242f-4745-a1c4-f960393ea712 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:44.119830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc0ad0-242f-4745-a1c4-f960393ea712 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:44.120146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc0ad0-242f-4745-a1c4-f960393ea712 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:44.120420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc0ad0-242f-4745-a1c4-f960393ea712 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:44.130449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc0ad0-242f-4745-a1c4-f960393ea712 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:44.132513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc0ad0-242f-4745-a1c4-f960393ea712 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:44.142533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1002/1984] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:44 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:50.618547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f842d-81cd-48ef-860c-cc91a01e7200 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:50.620040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2b1f842d-81cd-48ef-860c-cc91a01e7200 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:50.686900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f842d-81cd-48ef-860c-cc91a01e7200 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:50.691537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:50.692199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:50.692477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:50.692802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] PATH_INFO: `/v3/users/4dee7384bb1144eab057ab14462c30ae` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:50.693929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dee7384bb1144eab057ab14462c30ae)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:50.696602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:50.823139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dee7384bb1144eab057ab14462c30ae'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:50.871929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dee7384bb1144eab057ab14462c30ae'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:50.885615 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dee7384bb1144eab057ab14462c30ae'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:50.886306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dee7384bb1144eab057ab14462c30ae'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:50.886915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dee7384bb1144eab057ab14462c30ae'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:50.901808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 983/1985] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:50 2022] DELETE /identity/v3/users/4dee7384bb1144eab057ab14462c30ae => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:50.906521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b34e4a-5e16-4ebe-ac2d-7de6ce6a16ac None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:50.907110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b34e4a-5e16-4ebe-ac2d-7de6ce6a16ac None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:50.907311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b34e4a-5e16-4ebe-ac2d-7de6ce6a16ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:50.926354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d2b34e4a-5e16-4ebe-ac2d-7de6ce6a16ac None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:50.927555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d2b34e4a-5e16-4ebe-ac2d-7de6ce6a16ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:51.002188 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1003/1986] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:51.012680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:51.014294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:51.074841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ba144-3ea0-485c-a0ea-bd1ece7d8f79 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:51.078783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebef32-93f4-4238-9d98-e88cf77513f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:51.079393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebef32-93f4-4238-9d98-e88cf77513f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:51.079670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebef32-93f4-4238-9d98-e88cf77513f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:51.079937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebef32-93f4-4238-9d98-e88cf77513f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:51.089252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebef32-93f4-4238-9d98-e88cf77513f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:51.090676 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebef32-93f4-4238-9d98-e88cf77513f9 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:51.099998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 984/1987] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:51.474064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b34e4a-5e16-4ebe-ac2d-7de6ce6a16ac None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:51.475492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d2b34e4a-5e16-4ebe-ac2d-7de6ce6a16ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:51.490642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b34e4a-5e16-4ebe-ac2d-7de6ce6a16ac None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:51.495912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:51.496672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:51.496953 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:51.497223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] PATH_INFO: `/v3/projects/3bd432faa5484aeb9a341f460c638bf9` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:51.498388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd432faa5484aeb9a341f460c638bf9)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:51.500508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:51.604770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd432faa5484aeb9a341f460c638bf9', 'request_id': 'req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:51.611579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd432faa5484aeb9a341f460c638bf9', 'request_id': 'req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:51.666361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd432faa5484aeb9a341f460c638bf9', 'request_id': 'req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:51.677178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] The token cache is being invalidate because project 3bd432faa5484aeb9a341f460c638bf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:51.677471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:51.678116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:51.679595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1004/1988] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:51 2022] DELETE /identity/v3/projects/3bd432faa5484aeb9a341f460c638bf9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:51.684525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebef32-93f4-4238-9d98-e88cf77513f9 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:51.694182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebef32-93f4-4238-9d98-e88cf77513f9 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:51.712232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:51.751001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff46aeb-8aa8-4a8b-85f7-df89102ed116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:51.751653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff46aeb-8aa8-4a8b-85f7-df89102ed116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:51.751938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff46aeb-8aa8-4a8b-85f7-df89102ed116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:51.752252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff46aeb-8aa8-4a8b-85f7-df89102ed116 admin admin] PATH_INFO: `/v3/users/0a4e29fb91984a83b1af28581104e2a6` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:51.753388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff46aeb-8aa8-4a8b-85f7-df89102ed116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a4e29fb91984a83b1af28581104e2a6)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:51.755308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff46aeb-8aa8-4a8b-85f7-df89102ed116 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:51.879029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5ff46aeb-8aa8-4a8b-85f7-df89102ed116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a4e29fb91984a83b1af28581104e2a6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:51.957310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5ff46aeb-8aa8-4a8b-85f7-df89102ed116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a4e29fb91984a83b1af28581104e2a6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:51.970611 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5ff46aeb-8aa8-4a8b-85f7-df89102ed116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a4e29fb91984a83b1af28581104e2a6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:51.971200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5ff46aeb-8aa8-4a8b-85f7-df89102ed116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a4e29fb91984a83b1af28581104e2a6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:51.971803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5ff46aeb-8aa8-4a8b-85f7-df89102ed116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a4e29fb91984a83b1af28581104e2a6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:51.985631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 985/1989] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:51 2022] DELETE /identity/v3/users/0a4e29fb91984a83b1af28581104e2a6 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:52.433406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:52.434933 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:52.450379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b9a9e-92b2-42fa-afdc-60eaf3aee882 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:52.506935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:52.507485 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:52.507762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:52.508600 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] PATH_INFO: `/v3/projects/533d61ba28de40ccbc53168f370d790d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:52.509726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] RBAC: Authorizing `identity:delete_project(project_id=533d61ba28de40ccbc53168f370d790d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:52.511629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:52.522738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:52.615945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '533d61ba28de40ccbc53168f370d790d', 'request_id': 'req-9a3e0063-f0da-4508-990d-8c38b504296a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:52.622394 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '533d61ba28de40ccbc53168f370d790d', 'request_id': 'req-9a3e0063-f0da-4508-990d-8c38b504296a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:52.694782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '533d61ba28de40ccbc53168f370d790d', 'request_id': 'req-9a3e0063-f0da-4508-990d-8c38b504296a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:52.706261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] The token cache is being invalidate because project 533d61ba28de40ccbc53168f370d790d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:52.706556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:52.707215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9a3e0063-f0da-4508-990d-8c38b504296a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:52.709432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1005/1990] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:52 2022] DELETE /identity/v3/projects/533d61ba28de40ccbc53168f370d790d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:52.714473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57fb0f-6b7a-4c54-a422-e37e96c0306f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:52.715349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57fb0f-6b7a-4c54-a422-e37e96c0306f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:52.715834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57fb0f-6b7a-4c54-a422-e37e96c0306f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:52.736300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-af57fb0f-6b7a-4c54-a422-e37e96c0306f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:52.737502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-af57fb0f-6b7a-4c54-a422-e37e96c0306f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:52.801964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 986/1991] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:52.811395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfd8ba-38b5-4cd7-bf79-fddd21f79a9f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:52.811682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfd8ba-38b5-4cd7-bf79-fddd21f79a9f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:52.811960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfd8ba-38b5-4cd7-bf79-fddd21f79a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:52.831259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2ecfd8ba-38b5-4cd7-bf79-fddd21f79a9f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:52.832504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2ecfd8ba-38b5-4cd7-bf79-fddd21f79a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:52.841190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1006/1992] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:52.845406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57fb0f-6b7a-4c54-a422-e37e96c0306f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:52.855988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57fb0f-6b7a-4c54-a422-e37e96c0306f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:52.860659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab1320-2f8a-4503-9a1a-48aa7c78b480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:52.861252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab1320-2f8a-4503-9a1a-48aa7c78b480 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:52.861534 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab1320-2f8a-4503-9a1a-48aa7c78b480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:52.861838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab1320-2f8a-4503-9a1a-48aa7c78b480 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:52.862485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab1320-2f8a-4503-9a1a-48aa7c78b480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:38:52.862791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab1320-2f8a-4503-9a1a-48aa7c78b480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:52.864414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab1320-2f8a-4503-9a1a-48aa7c78b480 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:52.876086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 987/1993] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:38:52.880280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfd8ba-38b5-4cd7-bf79-fddd21f79a9f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:52.889722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfd8ba-38b5-4cd7-bf79-fddd21f79a9f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:52.894307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479e696-f9c5-4dfc-b013-cfab4b7cb98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:52.894895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479e696-f9c5-4dfc-b013-cfab4b7cb98a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:52.895171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479e696-f9c5-4dfc-b013-cfab4b7cb98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:52.895443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479e696-f9c5-4dfc-b013-cfab4b7cb98a admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:52.896087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479e696-f9c5-4dfc-b013-cfab4b7cb98a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:52.897968 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479e696-f9c5-4dfc-b013-cfab4b7cb98a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:52.982324 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1007/1994] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:52 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:52.986555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab1320-2f8a-4503-9a1a-48aa7c78b480 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:52.996994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab1320-2f8a-4503-9a1a-48aa7c78b480 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:53.023412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:53.056298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02912714-2395-4d18-a044-63e72843dc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:53.056958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912714-2395-4d18-a044-63e72843dc24 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:53.057232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912714-2395-4d18-a044-63e72843dc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:53.057501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912714-2395-4d18-a044-63e72843dc24 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:53.058148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02912714-2395-4d18-a044-63e72843dc24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:53.060270 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02912714-2395-4d18-a044-63e72843dc24 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:53.157526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 988/1995] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:52 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:53.161620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479e696-f9c5-4dfc-b013-cfab4b7cb98a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:53.172347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479e696-f9c5-4dfc-b013-cfab4b7cb98a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:53.177057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceaa49-777e-483f-9f11-d22a2aac36d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:53.177661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceaa49-777e-483f-9f11-d22a2aac36d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:53.177941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceaa49-777e-483f-9f11-d22a2aac36d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:53.178215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceaa49-777e-483f-9f11-d22a2aac36d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:53.178953 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceaa49-777e-483f-9f11-d22a2aac36d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:53.180694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceaa49-777e-483f-9f11-d22a2aac36d5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:53.192044 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1008/1996] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:53.194813 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02912714-2395-4d18-a044-63e72843dc24 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:53.205963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02912714-2395-4d18-a044-63e72843dc24 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:53.210572 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11135c59-45a1-41d7-8dad-000c72f8ee97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:53.211927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11135c59-45a1-41d7-8dad-000c72f8ee97 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:53.212293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11135c59-45a1-41d7-8dad-000c72f8ee97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:53.212575 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11135c59-45a1-41d7-8dad-000c72f8ee97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:53.213225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11135c59-45a1-41d7-8dad-000c72f8ee97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:53.214809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11135c59-45a1-41d7-8dad-000c72f8ee97 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:53.224947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 989/1997] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:53.229377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceaa49-777e-483f-9f11-d22a2aac36d5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:53.240032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceaa49-777e-483f-9f11-d22a2aac36d5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:53.244926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609274e6-e4de-4a9d-aba6-929441b6326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:53.245657 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609274e6-e4de-4a9d-aba6-929441b6326f admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:53.245939 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609274e6-e4de-4a9d-aba6-929441b6326f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:53.246215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609274e6-e4de-4a9d-aba6-929441b6326f admin admin] PATH_INFO: `/v3/projects/1bc34debfcb04730b422d6bd7c293230/users/6a8f72614f42443f8b21e359890dfe2a/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:53.262219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609274e6-e4de-4a9d-aba6-929441b6326f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc34debfcb04730b422d6bd7c293230, user_id=6a8f72614f42443f8b21e359890dfe2a, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:53.265123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609274e6-e4de-4a9d-aba6-929441b6326f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:53.352305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1009/1998] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:53 2022] PUT /identity/v3/projects/1bc34debfcb04730b422d6bd7c293230/users/6a8f72614f42443f8b21e359890dfe2a/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:53.363947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f8378-5a22-4f19-a90e-37f6749361d4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:53.364266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f8378-5a22-4f19-a90e-37f6749361d4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:53.364547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f8378-5a22-4f19-a90e-37f6749361d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:53.392838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-691f8378-5a22-4f19-a90e-37f6749361d4 None None] MFA Rules not processed for user `6a8f72614f42443f8b21e359890dfe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:53.394046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-691f8378-5a22-4f19-a90e-37f6749361d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:53.458460 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 990/1999] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:53.463226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609274e6-e4de-4a9d-aba6-929441b6326f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:53.523504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609274e6-e4de-4a9d-aba6-929441b6326f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:53.528329 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421dfff-3f28-4873-b440-1e3155f1b45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:53.528923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421dfff-3f28-4873-b440-1e3155f1b45a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:53.529200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421dfff-3f28-4873-b440-1e3155f1b45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:53.529472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421dfff-3f28-4873-b440-1e3155f1b45a admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:53.530085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421dfff-3f28-4873-b440-1e3155f1b45a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:53.531976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421dfff-3f28-4873-b440-1e3155f1b45a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:53.625375 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1010/2000] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:53 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:53.627507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f8378-5a22-4f19-a90e-37f6749361d4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:53.641635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f8378-5a22-4f19-a90e-37f6749361d4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:53.727493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f3ad5-3cc1-4ed6-9a19-b7792874b75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:53.728629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f3ad5-3cc1-4ed6-9a19-b7792874b75e admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:53.729133 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f3ad5-3cc1-4ed6-9a19-b7792874b75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:53.729595 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f3ad5-3cc1-4ed6-9a19-b7792874b75e admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:53.730724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f3ad5-3cc1-4ed6-9a19-b7792874b75e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:53.734467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f3ad5-3cc1-4ed6-9a19-b7792874b75e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:53.838600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 991/2001] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:53 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:53.842828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421dfff-3f28-4873-b440-1e3155f1b45a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:53.854319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421dfff-3f28-4873-b440-1e3155f1b45a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:53.859759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7e7a5-11ad-4857-bdf1-79e22701b08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:53.860219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e7a5-11ad-4857-bdf1-79e22701b08a admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:53.860528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e7a5-11ad-4857-bdf1-79e22701b08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:53.860804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e7a5-11ad-4857-bdf1-79e22701b08a admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:53.861460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7e7a5-11ad-4857-bdf1-79e22701b08a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:53.863868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7e7a5-11ad-4857-bdf1-79e22701b08a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:53.880818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1011/2002] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:53.885382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f3ad5-3cc1-4ed6-9a19-b7792874b75e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:53.901118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f3ad5-3cc1-4ed6-9a19-b7792874b75e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:53.906125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b86f3-970d-4f02-b253-fe4d6cd14aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:53.906721 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b86f3-970d-4f02-b253-fe4d6cd14aba admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:53.906973 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b86f3-970d-4f02-b253-fe4d6cd14aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:53.907248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b86f3-970d-4f02-b253-fe4d6cd14aba admin admin] PATH_INFO: `/v3/projects/6f8759289d6e4e3a8667fe5fe18672b3/users/a7131a9bb2bf4d40bb1513219609a96f/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:53.923201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b86f3-970d-4f02-b253-fe4d6cd14aba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f8759289d6e4e3a8667fe5fe18672b3, user_id=a7131a9bb2bf4d40bb1513219609a96f, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:53.925917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b86f3-970d-4f02-b253-fe4d6cd14aba admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:53.993482 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 992/2003] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:53 2022] PUT /identity/v3/projects/6f8759289d6e4e3a8667fe5fe18672b3/users/a7131a9bb2bf4d40bb1513219609a96f/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:54.005805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7e7a5-11ad-4857-bdf1-79e22701b08a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:54.020844 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7e7a5-11ad-4857-bdf1-79e22701b08a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:54.030946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:54.083451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676e1e1-adfb-43f9-8d52-1de29660ef94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:54.084105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676e1e1-adfb-43f9-8d52-1de29660ef94 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:54.084362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676e1e1-adfb-43f9-8d52-1de29660ef94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:54.084633 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676e1e1-adfb-43f9-8d52-1de29660ef94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:54.085274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676e1e1-adfb-43f9-8d52-1de29660ef94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:54.088000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676e1e1-adfb-43f9-8d52-1de29660ef94 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:54.098364 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1012/2004] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:54.104335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b86f3-970d-4f02-b253-fe4d6cd14aba admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:54.110430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:54.115934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b86f3-970d-4f02-b253-fe4d6cd14aba admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:54.121624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811bbb4-a99a-4061-81bd-0137e330b696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:54.122375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811bbb4-a99a-4061-81bd-0137e330b696 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:54.122663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811bbb4-a99a-4061-81bd-0137e330b696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:54.122944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811bbb4-a99a-4061-81bd-0137e330b696 admin admin] PATH_INFO: `/v3/projects/6f8759289d6e4e3a8667fe5fe18672b3/users/a7131a9bb2bf4d40bb1513219609a96f/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:54.125104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811bbb4-a99a-4061-81bd-0137e330b696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f8759289d6e4e3a8667fe5fe18672b3, user_id=a7131a9bb2bf4d40bb1513219609a96f, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:54.127856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811bbb4-a99a-4061-81bd-0137e330b696 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:54.139183 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 993/2005] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:54 2022] PUT /identity/v3/projects/6f8759289d6e4e3a8667fe5fe18672b3/users/a7131a9bb2bf4d40bb1513219609a96f/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:54.150182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8e341-72ec-4e9d-b4e0-b407ddc0764e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:54.150469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8e341-72ec-4e9d-b4e0-b407ddc0764e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:54.150802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8e341-72ec-4e9d-b4e0-b407ddc0764e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:54.180689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4ba8e341-72ec-4e9d-b4e0-b407ddc0764e None None] MFA Rules not processed for user `a7131a9bb2bf4d40bb1513219609a96f`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:54.181977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4ba8e341-72ec-4e9d-b4e0-b407ddc0764e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:54.256573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1013/2006] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:54.268401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811bbb4-a99a-4061-81bd-0137e330b696 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:54.270147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7811bbb4-a99a-4061-81bd-0137e330b696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:54.359641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811bbb4-a99a-4061-81bd-0137e330b696 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:54.364498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333ce2c-23ae-4f5f-99b0-6f9fdc05ef70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:54.365481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333ce2c-23ae-4f5f-99b0-6f9fdc05ef70 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:54.365955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333ce2c-23ae-4f5f-99b0-6f9fdc05ef70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:54.366472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333ce2c-23ae-4f5f-99b0-6f9fdc05ef70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:54.385998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333ce2c-23ae-4f5f-99b0-6f9fdc05ef70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:54.388485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333ce2c-23ae-4f5f-99b0-6f9fdc05ef70 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:54.404528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 994/2007] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:54.440853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8e341-72ec-4e9d-b4e0-b407ddc0764e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:54.543755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8e341-72ec-4e9d-b4e0-b407ddc0764e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:54.548499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:54.549223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:54.549562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:54.549874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] PATH_INFO: `/v3/users/6a8f72614f42443f8b21e359890dfe2a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:54.551058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a8f72614f42443f8b21e359890dfe2a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:54.553062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:54.734143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:54.744455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a8f72614f42443f8b21e359890dfe2a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:54.833269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a8f72614f42443f8b21e359890dfe2a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:54.851746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a8f72614f42443f8b21e359890dfe2a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:54.859539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a8f72614f42443f8b21e359890dfe2a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:54.860311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a8f72614f42443f8b21e359890dfe2a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:54.868202 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1014/2008] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:54 2022] DELETE /identity/v3/users/6a8f72614f42443f8b21e359890dfe2a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:54.873702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd758f-c219-48dd-b1a3-3d1ef9b55586 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:54.874603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd758f-c219-48dd-b1a3-3d1ef9b55586 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:54.874895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd758f-c219-48dd-b1a3-3d1ef9b55586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:54.896359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8bfd758f-c219-48dd-b1a3-3d1ef9b55586 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:54.897618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8bfd758f-c219-48dd-b1a3-3d1ef9b55586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:54.975317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 995/2009] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:54.987859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:54.990569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:55.056745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d71a39-b3aa-408c-855f-2f79a383cedc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:55.060497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eb207-7998-45f6-a46a-6260473c3f9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:55.061240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eb207-7998-45f6-a46a-6260473c3f9a service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:55.061517 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eb207-7998-45f6-a46a-6260473c3f9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:55.061789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eb207-7998-45f6-a46a-6260473c3f9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:55.082229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40eb207-7998-45f6-a46a-6260473c3f9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:55.084280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40eb207-7998-45f6-a46a-6260473c3f9a service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:55.095787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1015/2010] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:55.439263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd758f-c219-48dd-b1a3-3d1ef9b55586 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:55.440926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8bfd758f-c219-48dd-b1a3-3d1ef9b55586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:55.456610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd758f-c219-48dd-b1a3-3d1ef9b55586 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:55.465786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6774d-1615-4d8b-be7a-d38b01be55dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:55.466559 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6774d-1615-4d8b-be7a-d38b01be55dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:55.466842 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6774d-1615-4d8b-be7a-d38b01be55dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:55.467107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6774d-1615-4d8b-be7a-d38b01be55dc admin admin] PATH_INFO: `/v3/users/aaf81fb89c8342ed92216594913ca604` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:55.468296 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6774d-1615-4d8b-be7a-d38b01be55dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaf81fb89c8342ed92216594913ca604)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:55.470241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6774d-1615-4d8b-be7a-d38b01be55dc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:55.677813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eb207-7998-45f6-a46a-6260473c3f9a service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:55.679410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d40eb207-7998-45f6-a46a-6260473c3f9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:55.692315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f0b6774d-1615-4d8b-be7a-d38b01be55dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaf81fb89c8342ed92216594913ca604'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:55.698274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eb207-7998-45f6-a46a-6260473c3f9a service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:55.703890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:55.704673 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:55.704956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:55.705237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] PATH_INFO: `/v3/projects/1bc34debfcb04730b422d6bd7c293230` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:55.706525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc34debfcb04730b422d6bd7c293230)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:55.708610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:55.766310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f0b6774d-1615-4d8b-be7a-d38b01be55dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaf81fb89c8342ed92216594913ca604'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:55.788870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f0b6774d-1615-4d8b-be7a-d38b01be55dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaf81fb89c8342ed92216594913ca604'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:55.790442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f0b6774d-1615-4d8b-be7a-d38b01be55dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaf81fb89c8342ed92216594913ca604'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:55.791388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f0b6774d-1615-4d8b-be7a-d38b01be55dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaf81fb89c8342ed92216594913ca604'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:55.810903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 996/2011] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:55 2022] DELETE /identity/v3/users/aaf81fb89c8342ed92216594913ca604 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:55.815287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc569f-5bda-42ed-ab31-e2c902656cdc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:55.815561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc569f-5bda-42ed-ab31-e2c902656cdc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:55.815840 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc569f-5bda-42ed-ab31-e2c902656cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:55.824104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:55.833845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc34debfcb04730b422d6bd7c293230', 'request_id': 'req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:55.834901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-31dc569f-5bda-42ed-ab31-e2c902656cdc None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:55.836819 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-31dc569f-5bda-42ed-ab31-e2c902656cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:55.841171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc34debfcb04730b422d6bd7c293230', 'request_id': 'req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:55.896945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 997/2012] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:55.908796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc569f-5bda-42ed-ab31-e2c902656cdc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:55.910355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-31dc569f-5bda-42ed-ab31-e2c902656cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:55.947787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc34debfcb04730b422d6bd7c293230', 'request_id': 'req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:55.963014 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] The token cache is being invalidate because project 1bc34debfcb04730b422d6bd7c293230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:55.963483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:55.964438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:55.966084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1016/2013] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:55 2022] DELETE /identity/v3/projects/1bc34debfcb04730b422d6bd7c293230 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:55.978341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc569f-5bda-42ed-ab31-e2c902656cdc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:55.983836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdadb8-156a-463e-80e7-a2422c92a441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:55.993209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdadb8-156a-463e-80e7-a2422c92a441 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:55.993209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdadb8-156a-463e-80e7-a2422c92a441 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:55.993209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdadb8-156a-463e-80e7-a2422c92a441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:55.993584 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:55.993682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2acdadb8-156a-463e-80e7-a2422c92a441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:55.993782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:56.002709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acdadb8-156a-463e-80e7-a2422c92a441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:56.004250 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acdadb8-156a-463e-80e7-a2422c92a441 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:56.012494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21088c7-f21c-4f6b-8f73-d2eb9f90b8d3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:56.017294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 998/2014] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:38:56.018533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:56.019270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:56.019538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:56.020031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] PATH_INFO: `/v3/users/a7131a9bb2bf4d40bb1513219609a96f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:56.021230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7131a9bb2bf4d40bb1513219609a96f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:56.023221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:56.205150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:56.220120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7131a9bb2bf4d40bb1513219609a96f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:56.300583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7131a9bb2bf4d40bb1513219609a96f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:56.315065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7131a9bb2bf4d40bb1513219609a96f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:56.331211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7131a9bb2bf4d40bb1513219609a96f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:56.332607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7131a9bb2bf4d40bb1513219609a96f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:56.341531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1017/2015] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:55 2022] DELETE /identity/v3/users/a7131a9bb2bf4d40bb1513219609a96f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:56.484224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdadb8-156a-463e-80e7-a2422c92a441 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:56.485775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2acdadb8-156a-463e-80e7-a2422c92a441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:56.510814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdadb8-156a-463e-80e7-a2422c92a441 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:56.576425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:56.577060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:56.577336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:56.577718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] PATH_INFO: `/v3/projects/46be5db93569499fbe8169151eca4e1b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:56.578743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46be5db93569499fbe8169151eca4e1b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:56.580785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:56.777363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46be5db93569499fbe8169151eca4e1b', 'request_id': 'req-1b1984f3-e72d-4214-9a03-98f272604473'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:56.779269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:56.786035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46be5db93569499fbe8169151eca4e1b', 'request_id': 'req-1b1984f3-e72d-4214-9a03-98f272604473'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:56.848581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:56.848581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:56.868731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e549a-6edd-43fc-a057-6f940ee0c022 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:56.900822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46be5db93569499fbe8169151eca4e1b', 'request_id': 'req-1b1984f3-e72d-4214-9a03-98f272604473'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:56.914805 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] The token cache is being invalidate because project 46be5db93569499fbe8169151eca4e1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:56.915214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:56.915926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:56.917661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 999/2016] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:56 2022] DELETE /identity/v3/projects/46be5db93569499fbe8169151eca4e1b => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:56.921891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:56.923317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:56.963675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:56.964455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:56.964807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:56.965158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] PATH_INFO: `/v3/projects/6f8759289d6e4e3a8667fe5fe18672b3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:56.966434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f8759289d6e4e3a8667fe5fe18672b3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:56.968485 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:57.031900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1984f3-e72d-4214-9a03-98f272604473 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:57.037226 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:57.037867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:57.038147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:57.038422 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] PATH_INFO: `/v3/users/654d0760846846819d5d3dc8ee092ea4` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:57.040317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=654d0760846846819d5d3dc8ee092ea4)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:57.042424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:57.110497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f8759289d6e4e3a8667fe5fe18672b3', 'request_id': 'req-5fbcea62-3fe8-4979-afe3-9308053b1710'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:57.117773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f8759289d6e4e3a8667fe5fe18672b3', 'request_id': 'req-5fbcea62-3fe8-4979-afe3-9308053b1710'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:57.197635 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f8759289d6e4e3a8667fe5fe18672b3', 'request_id': 'req-5fbcea62-3fe8-4979-afe3-9308053b1710'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:57.208731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] The token cache is being invalidate because project 6f8759289d6e4e3a8667fe5fe18672b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:57.208903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:57.209566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5fbcea62-3fe8-4979-afe3-9308053b1710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:57.211169 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1018/2017] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:56 2022] DELETE /identity/v3/projects/6f8759289d6e4e3a8667fe5fe18672b3 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:57.215998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565a424-0bba-4b76-9d51-19d340eed879 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:57.216316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565a424-0bba-4b76-9d51-19d340eed879 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:57.216598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565a424-0bba-4b76-9d51-19d340eed879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:57.234473 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '654d0760846846819d5d3dc8ee092ea4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:57.239374 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6565a424-0bba-4b76-9d51-19d340eed879 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:57.240630 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6565a424-0bba-4b76-9d51-19d340eed879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:57.281610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:57.302119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '654d0760846846819d5d3dc8ee092ea4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:57.321395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1019/2018] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:57.328154 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165b9bb-84e6-4b71-8d80-14c4ef069a84 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:57.328445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165b9bb-84e6-4b71-8d80-14c4ef069a84 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:57.328723 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165b9bb-84e6-4b71-8d80-14c4ef069a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:57.341864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '654d0760846846819d5d3dc8ee092ea4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:57.343318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '654d0760846846819d5d3dc8ee092ea4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:57.344073 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '654d0760846846819d5d3dc8ee092ea4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:57.351221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7165b9bb-84e6-4b71-8d80-14c4ef069a84 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:57.356059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7165b9bb-84e6-4b71-8d80-14c4ef069a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:57.357877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1000/2019] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:56 2022] DELETE /identity/v3/users/654d0760846846819d5d3dc8ee092ea4 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:57.361029 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1020/2020] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:57.365048 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:57.432828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a094d-61c7-4322-8e57-6b131ece72dd admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:57.437671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245983d-5c69-48e7-9e9b-3896ad1c6d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:57.438275 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245983d-5c69-48e7-9e9b-3896ad1c6d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:57.438582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245983d-5c69-48e7-9e9b-3896ad1c6d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:57.438854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245983d-5c69-48e7-9e9b-3896ad1c6d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:57.439505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245983d-5c69-48e7-9e9b-3896ad1c6d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:38:57.439803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245983d-5c69-48e7-9e9b-3896ad1c6d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:57.441429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245983d-5c69-48e7-9e9b-3896ad1c6d9c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:57.457585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1001/2021] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:38:57.459563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165b9bb-84e6-4b71-8d80-14c4ef069a84 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:57.468296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165b9bb-84e6-4b71-8d80-14c4ef069a84 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:57.473170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5915a-775d-4674-a8ed-92388d2d836e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:57.473761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5915a-775d-4674-a8ed-92388d2d836e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:57.474037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5915a-775d-4674-a8ed-92388d2d836e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:57.474305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5915a-775d-4674-a8ed-92388d2d836e admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:57.474936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5915a-775d-4674-a8ed-92388d2d836e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:57.477471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5915a-775d-4674-a8ed-92388d2d836e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:57.590905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1021/2022] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:57 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:57.595077 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245983d-5c69-48e7-9e9b-3896ad1c6d9c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:57.605835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:57.612521 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245983d-5c69-48e7-9e9b-3896ad1c6d9c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:57.683727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4e09d-7cb1-40dc-8f02-9fbd541e2f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:57.684391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4e09d-7cb1-40dc-8f02-9fbd541e2f98 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:57.687571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4e09d-7cb1-40dc-8f02-9fbd541e2f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:57.687571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4e09d-7cb1-40dc-8f02-9fbd541e2f98 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:57.687571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4e09d-7cb1-40dc-8f02-9fbd541e2f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:57.687941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4e09d-7cb1-40dc-8f02-9fbd541e2f98 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:57.785313 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1002/2023] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:57 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:38:57.786293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5915a-775d-4674-a8ed-92388d2d836e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:57.797222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5915a-775d-4674-a8ed-92388d2d836e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:57.804351 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ba476-6550-4ff4-aee7-50f4b1b7c393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:57.804351 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ba476-6550-4ff4-aee7-50f4b1b7c393 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:57.804351 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ba476-6550-4ff4-aee7-50f4b1b7c393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:57.810268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ba476-6550-4ff4-aee7-50f4b1b7c393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:57.810268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ba476-6550-4ff4-aee7-50f4b1b7c393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:57.810268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ba476-6550-4ff4-aee7-50f4b1b7c393 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:57.812667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4e09d-7cb1-40dc-8f02-9fbd541e2f98 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:57.814980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d1a4e09d-7cb1-40dc-8f02-9fbd541e2f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:57.831891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4e09d-7cb1-40dc-8f02-9fbd541e2f98 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:57.833286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1022/2024] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:57.845573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:57.846239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:57.846528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:57.846817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] PATH_INFO: `/v3/projects/65b8a1b404144673ade05d3245ab3b87` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:57.847587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ba476-6550-4ff4-aee7-50f4b1b7c393 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:57.848199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b8a1b404144673ade05d3245ab3b87)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:57.850523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:57.863194 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ba476-6550-4ff4-aee7-50f4b1b7c393 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:57.868938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73e8c5-ef03-47aa-bbea-161cc3395421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:57.869548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e8c5-ef03-47aa-bbea-161cc3395421 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:57.869872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e8c5-ef03-47aa-bbea-161cc3395421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:57.870144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e8c5-ef03-47aa-bbea-161cc3395421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:57.870949 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73e8c5-ef03-47aa-bbea-161cc3395421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:57.872739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73e8c5-ef03-47aa-bbea-161cc3395421 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:57.896767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1023/2025] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:57.899741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73e8c5-ef03-47aa-bbea-161cc3395421 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:57.915731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73e8c5-ef03-47aa-bbea-161cc3395421 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:57.915731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdafe23-c90b-4233-a88c-4881b42594a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:57.916388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdafe23-c90b-4233-a88c-4881b42594a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:57.916669 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdafe23-c90b-4233-a88c-4881b42594a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:57.917031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdafe23-c90b-4233-a88c-4881b42594a1 admin admin] PATH_INFO: `/v3/projects/e13d5f91ab564825bac7c55831d7d027/users/4580f6c80ea24e38a924bc1d627a2b78/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:57.933853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdafe23-c90b-4233-a88c-4881b42594a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13d5f91ab564825bac7c55831d7d027, user_id=4580f6c80ea24e38a924bc1d627a2b78, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:57.940981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdafe23-c90b-4233-a88c-4881b42594a1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:58.061423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1024/2026] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:57 2022] PUT /identity/v3/projects/e13d5f91ab564825bac7c55831d7d027/users/4580f6c80ea24e38a924bc1d627a2b78/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:58.070027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b8a1b404144673ade05d3245ab3b87', 'request_id': 'req-5dbc51e9-9806-4cde-9041-77ec2e480f9d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:58.074550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b521d4-8685-4895-ad47-e573b291427c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:58.074848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b521d4-8685-4895-ad47-e573b291427c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:58.075140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b521d4-8685-4895-ad47-e573b291427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:58.078340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b8a1b404144673ade05d3245ab3b87', 'request_id': 'req-5dbc51e9-9806-4cde-9041-77ec2e480f9d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:58.105824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-82b521d4-8685-4895-ad47-e573b291427c None None] MFA Rules not processed for user `4580f6c80ea24e38a924bc1d627a2b78`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:58.107064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-82b521d4-8685-4895-ad47-e573b291427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:58.170391 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1025/2027] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:38:58.174937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b521d4-8685-4895-ad47-e573b291427c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:58.209435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b8a1b404144673ade05d3245ab3b87', 'request_id': 'req-5dbc51e9-9806-4cde-9041-77ec2e480f9d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:58.221758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] The token cache is being invalidate because project 65b8a1b404144673ade05d3245ab3b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:38:58.222189 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:58.222983 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:58.234825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1003/2028] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:38:57 2022] DELETE /identity/v3/projects/65b8a1b404144673ade05d3245ab3b87 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:38:58.250522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b521d4-8685-4895-ad47-e573b291427c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:58.256799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b0d18-be3e-432b-84d5-7cdbdae0000a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:58.256958 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b0d18-be3e-432b-84d5-7cdbdae0000a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:58.257195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b0d18-be3e-432b-84d5-7cdbdae0000a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:58.258461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b0d18-be3e-432b-84d5-7cdbdae0000a admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:58.259621 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b0d18-be3e-432b-84d5-7cdbdae0000a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:58.262187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b0d18-be3e-432b-84d5-7cdbdae0000a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:58.376792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1026/2029] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:38:58 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:58.377506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:58.379314 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:58.395734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc51e9-9806-4cde-9041-77ec2e480f9d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:58.454421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f08132-e763-4f67-8e8a-d9d34f4ac6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:58.455029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f08132-e763-4f67-8e8a-d9d34f4ac6dc admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:58.455317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f08132-e763-4f67-8e8a-d9d34f4ac6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:58.455597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f08132-e763-4f67-8e8a-d9d34f4ac6dc admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:58.456245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f08132-e763-4f67-8e8a-d9d34f4ac6dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:58.458201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f08132-e763-4f67-8e8a-d9d34f4ac6dc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:58.535147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1004/2030] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:38:58 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:38:58.539080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b0d18-be3e-432b-84d5-7cdbdae0000a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:58.549923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b0d18-be3e-432b-84d5-7cdbdae0000a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:58.555325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41572b07-4352-49b3-ab0f-f8fd92a4ed02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:58.555954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41572b07-4352-49b3-ab0f-f8fd92a4ed02 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:58.556277 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41572b07-4352-49b3-ab0f-f8fd92a4ed02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:58.556552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41572b07-4352-49b3-ab0f-f8fd92a4ed02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:58.557207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41572b07-4352-49b3-ab0f-f8fd92a4ed02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:58.558814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41572b07-4352-49b3-ab0f-f8fd92a4ed02 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:58.568814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1027/2031] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:58.573720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f08132-e763-4f67-8e8a-d9d34f4ac6dc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:58.583848 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f08132-e763-4f67-8e8a-d9d34f4ac6dc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:58.588997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa44ee-115e-4b57-b405-1cabef1e1c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:58.594780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa44ee-115e-4b57-b405-1cabef1e1c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:58.594780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa44ee-115e-4b57-b405-1cabef1e1c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:58.594780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa44ee-115e-4b57-b405-1cabef1e1c4b admin admin] PATH_INFO: `/v3/projects/74d82b222d544ba98fb4d4bdfdda8b13/users/340df16ccb4445deba5fcc0b5d326022/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:58.606300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa44ee-115e-4b57-b405-1cabef1e1c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d82b222d544ba98fb4d4bdfdda8b13, user_id=340df16ccb4445deba5fcc0b5d326022, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:58.609041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa44ee-115e-4b57-b405-1cabef1e1c4b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:58.695290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1005/2032] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:58 2022] PUT /identity/v3/projects/74d82b222d544ba98fb4d4bdfdda8b13/users/340df16ccb4445deba5fcc0b5d326022/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:58.699316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41572b07-4352-49b3-ab0f-f8fd92a4ed02 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:58.709326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41572b07-4352-49b3-ab0f-f8fd92a4ed02 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:58.713061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:58.769047 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f528792-5d62-4561-bf18-6951587bdfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:58.769650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f528792-5d62-4561-bf18-6951587bdfd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:58.769923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f528792-5d62-4561-bf18-6951587bdfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:58.770233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f528792-5d62-4561-bf18-6951587bdfd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:58.770910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f528792-5d62-4561-bf18-6951587bdfd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:58.772522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f528792-5d62-4561-bf18-6951587bdfd8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:58.786792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1028/2033] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:38:58.791782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa44ee-115e-4b57-b405-1cabef1e1c4b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:58.802666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa44ee-115e-4b57-b405-1cabef1e1c4b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:58.807716 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416fdd1-66cd-43eb-a96f-e48749f40517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:58.808341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416fdd1-66cd-43eb-a96f-e48749f40517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:58.808621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416fdd1-66cd-43eb-a96f-e48749f40517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:58.808915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416fdd1-66cd-43eb-a96f-e48749f40517 admin admin] PATH_INFO: `/v3/projects/74d82b222d544ba98fb4d4bdfdda8b13/users/340df16ccb4445deba5fcc0b5d326022/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:58.810901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416fdd1-66cd-43eb-a96f-e48749f40517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d82b222d544ba98fb4d4bdfdda8b13, user_id=340df16ccb4445deba5fcc0b5d326022, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:58.814905 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416fdd1-66cd-43eb-a96f-e48749f40517 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:58.824179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1006/2034] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:38:58 2022] PUT /identity/v3/projects/74d82b222d544ba98fb4d4bdfdda8b13/users/340df16ccb4445deba5fcc0b5d326022/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:58.834531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcda624-b3b5-4cf9-8732-c5cdf4143e48 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:58.838044 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcda624-b3b5-4cf9-8732-c5cdf4143e48 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:58.838044 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcda624-b3b5-4cf9-8732-c5cdf4143e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:58.867191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9bcda624-b3b5-4cf9-8732-c5cdf4143e48 None None] MFA Rules not processed for user `340df16ccb4445deba5fcc0b5d326022`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:58.868608 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9bcda624-b3b5-4cf9-8732-c5cdf4143e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:58.937928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1029/2035] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:38:58.953396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416fdd1-66cd-43eb-a96f-e48749f40517 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:58.953396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8416fdd1-66cd-43eb-a96f-e48749f40517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:59.018338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416fdd1-66cd-43eb-a96f-e48749f40517 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:59.024243 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:59.036467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f1351-c4a2-4649-8434-077461369139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:59.037153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f1351-c4a2-4649-8434-077461369139 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:59.037388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f1351-c4a2-4649-8434-077461369139 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:59.037645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f1351-c4a2-4649-8434-077461369139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:59.047324 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f1351-c4a2-4649-8434-077461369139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:59.048859 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f1351-c4a2-4649-8434-077461369139 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:59.058222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1007/2036] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:59.168398 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcda624-b3b5-4cf9-8732-c5cdf4143e48 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:59.175388 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9bcda624-b3b5-4cf9-8732-c5cdf4143e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:59.183131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcda624-b3b5-4cf9-8732-c5cdf4143e48 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:59.187067 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391d074-21b1-4068-bc24-13c27afc26a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:59.187752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391d074-21b1-4068-bc24-13c27afc26a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:59.188058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391d074-21b1-4068-bc24-13c27afc26a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:59.188340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391d074-21b1-4068-bc24-13c27afc26a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:59.190234 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a391d074-21b1-4068-bc24-13c27afc26a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:59.263320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391d074-21b1-4068-bc24-13c27afc26a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:59.264942 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391d074-21b1-4068-bc24-13c27afc26a3 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:59.275560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1030/2037] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:38:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:38:59.318553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f1351-c4a2-4649-8434-077461369139 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:38:59.320106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6d7f1351-c4a2-4649-8434-077461369139 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:59.434465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f1351-c4a2-4649-8434-077461369139 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:38:59.450004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:38:59.466932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:59.469767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:59.470719 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] PATH_INFO: `/v3/users/6f3d883d15d3429689e035d4b73b3a88` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:59.471987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f3d883d15d3429689e035d4b73b3a88)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:38:59.474319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:38:59.512973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:38:59.744849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f3d883d15d3429689e035d4b73b3a88'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:59.827057 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f3d883d15d3429689e035d4b73b3a88'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:59.845793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f3d883d15d3429689e035d4b73b3a88'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:59.846570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f3d883d15d3429689e035d4b73b3a88'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:59.847187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f3d883d15d3429689e035d4b73b3a88'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:38:59.864816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1008/2038] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:38:59 2022] DELETE /identity/v3/users/6f3d883d15d3429689e035d4b73b3a88 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:38:59.866706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7f86b-54d9-4e11-b44c-eaa3ef89e786 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:38:59.866995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7f86b-54d9-4e11-b44c-eaa3ef89e786 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:38:59.867857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7f86b-54d9-4e11-b44c-eaa3ef89e786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:38:59.889668 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-23f7f86b-54d9-4e11-b44c-eaa3ef89e786 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:38:59.891181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-23f7f86b-54d9-4e11-b44c-eaa3ef89e786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:38:59.929371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:00.016891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1031/2039] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:00.045196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:00.046646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:00.178333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551489b1-de72-4c21-ad22-324dea7d48e5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:00.183691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a8a2d-441a-4cee-92a2-1fb9242e5e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:00.184617 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a8a2d-441a-4cee-92a2-1fb9242e5e91 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:00.185131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a8a2d-441a-4cee-92a2-1fb9242e5e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:00.185402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a8a2d-441a-4cee-92a2-1fb9242e5e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:00.195950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a8a2d-441a-4cee-92a2-1fb9242e5e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:00.197893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a8a2d-441a-4cee-92a2-1fb9242e5e91 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:00.209553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1009/2040] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:00.959440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7f86b-54d9-4e11-b44c-eaa3ef89e786 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:00.961554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-23f7f86b-54d9-4e11-b44c-eaa3ef89e786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:00.980406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7f86b-54d9-4e11-b44c-eaa3ef89e786 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:00.986433 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:00.987184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:00.987520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:00.992497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] PATH_INFO: `/v3/projects/1a1a3b55ff2a419fa2a0507c05a9ec51` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:00.992497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a1a3b55ff2a419fa2a0507c05a9ec51)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:00.992497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:01.116585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a1a3b55ff2a419fa2a0507c05a9ec51', 'request_id': 'req-474076df-22bc-424a-bff5-25fb41e2727f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:01.123746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a1a3b55ff2a419fa2a0507c05a9ec51', 'request_id': 'req-474076df-22bc-424a-bff5-25fb41e2727f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:01.180809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a1a3b55ff2a419fa2a0507c05a9ec51', 'request_id': 'req-474076df-22bc-424a-bff5-25fb41e2727f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:01.197741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] The token cache is being invalidate because project 1a1a3b55ff2a419fa2a0507c05a9ec51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:01.198275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:01.199286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:01.205011 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1032/2041] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:00 2022] DELETE /identity/v3/projects/1a1a3b55ff2a419fa2a0507c05a9ec51 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:01.206162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a8a2d-441a-4cee-92a2-1fb9242e5e91 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:01.219098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a8a2d-441a-4cee-92a2-1fb9242e5e91 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:01.292602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:01.293537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:01.294210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:01.294509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] PATH_INFO: `/v3/users/1106ef2995cb4fa9a69a6c7135fcbc19` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:01.295722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1106ef2995cb4fa9a69a6c7135fcbc19)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:01.297736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:01.547241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1106ef2995cb4fa9a69a6c7135fcbc19'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:01.615450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1106ef2995cb4fa9a69a6c7135fcbc19'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:01.637603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1106ef2995cb4fa9a69a6c7135fcbc19'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:01.638440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1106ef2995cb4fa9a69a6c7135fcbc19'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:01.639140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1106ef2995cb4fa9a69a6c7135fcbc19'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:01.650345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:01.657619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1010/2042] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:01 2022] DELETE /identity/v3/users/1106ef2995cb4fa9a69a6c7135fcbc19 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:02.248833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:02.254397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:02.276773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474076df-22bc-424a-bff5-25fb41e2727f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:02.360705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:02.361342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:02.361700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:02.361973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] PATH_INFO: `/v3/projects/cd18b25144d74fd09d0769d86c2c64bf` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:02.363281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd18b25144d74fd09d0769d86c2c64bf)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:02.365502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:02.387812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:02.473188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:02.474698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:02.488842 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7a489-ae73-4613-b2d3-6bd29b4e8c1b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:02.494245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24ac6e-963b-42ff-a479-3a94d20714f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:02.494876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24ac6e-963b-42ff-a479-3a94d20714f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:02.495156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24ac6e-963b-42ff-a479-3a94d20714f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:02.495431 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24ac6e-963b-42ff-a479-3a94d20714f0 admin admin] PATH_INFO: `/v3/users/4580f6c80ea24e38a924bc1d627a2b78` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:02.496593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24ac6e-963b-42ff-a479-3a94d20714f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4580f6c80ea24e38a924bc1d627a2b78)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:02.498513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24ac6e-963b-42ff-a479-3a94d20714f0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:02.545209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd18b25144d74fd09d0769d86c2c64bf', 'request_id': 'req-3f383a92-bca8-4dc5-afcb-47f58c7a5189'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:02.551866 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd18b25144d74fd09d0769d86c2c64bf', 'request_id': 'req-3f383a92-bca8-4dc5-afcb-47f58c7a5189'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:02.615357 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd18b25144d74fd09d0769d86c2c64bf', 'request_id': 'req-3f383a92-bca8-4dc5-afcb-47f58c7a5189'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:02.627478 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] The token cache is being invalidate because project cd18b25144d74fd09d0769d86c2c64bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:02.627872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:02.637108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3f383a92-bca8-4dc5-afcb-47f58c7a5189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:02.642546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1033/2043] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:02 2022] DELETE /identity/v3/projects/cd18b25144d74fd09d0769d86c2c64bf => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:02.644200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97c269-de10-4721-b265-1a895accd7e7 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:02.644352 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97c269-de10-4721-b265-1a895accd7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:02.644693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97c269-de10-4721-b265-1a895accd7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:02.667320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bd97c269-de10-4721-b265-1a895accd7e7 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:02.668596 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bd97c269-de10-4721-b265-1a895accd7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:02.692239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6d24ac6e-963b-42ff-a479-3a94d20714f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4580f6c80ea24e38a924bc1d627a2b78'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:02.754307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1034/2044] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:02.762728 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b165f14-3c16-49ae-b4f0-847be9be7596 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:02.763634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b165f14-3c16-49ae-b4f0-847be9be7596 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:02.763634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b165f14-3c16-49ae-b4f0-847be9be7596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:02.780758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6d24ac6e-963b-42ff-a479-3a94d20714f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4580f6c80ea24e38a924bc1d627a2b78'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:02.783624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8b165f14-3c16-49ae-b4f0-847be9be7596 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:02.784937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8b165f14-3c16-49ae-b4f0-847be9be7596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:02.798143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1035/2045] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:02.798566 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6d24ac6e-963b-42ff-a479-3a94d20714f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4580f6c80ea24e38a924bc1d627a2b78'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:02.799150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b165f14-3c16-49ae-b4f0-847be9be7596 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:02.800125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6d24ac6e-963b-42ff-a479-3a94d20714f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4580f6c80ea24e38a924bc1d627a2b78'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:02.800951 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6d24ac6e-963b-42ff-a479-3a94d20714f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4580f6c80ea24e38a924bc1d627a2b78'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:02.809555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b165f14-3c16-49ae-b4f0-847be9be7596 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:02.818122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c07202-2758-4729-8118-366d37a4d1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:02.818808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c07202-2758-4729-8118-366d37a4d1dc admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:02.819091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c07202-2758-4729-8118-366d37a4d1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:02.819364 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c07202-2758-4729-8118-366d37a4d1dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:02.820284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c07202-2758-4729-8118-366d37a4d1dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:02.820773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c07202-2758-4729-8118-366d37a4d1dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:02.822694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c07202-2758-4729-8118-366d37a4d1dc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:02.834698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1011/2046] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:02 2022] DELETE /identity/v3/users/4580f6c80ea24e38a924bc1d627a2b78 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:02.834698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c0e69-cb13-43d0-9309-f71091ec94d5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:02.834698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c0e69-cb13-43d0-9309-f71091ec94d5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:02.834698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c0e69-cb13-43d0-9309-f71091ec94d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:02.836550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1036/2047] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:02.841366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c07202-2758-4729-8118-366d37a4d1dc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:02.843328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-78c07202-2758-4729-8118-366d37a4d1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:02.856343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2e0c0e69-cb13-43d0-9309-f71091ec94d5 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:02.859671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2e0c0e69-cb13-43d0-9309-f71091ec94d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:02.865977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c07202-2758-4729-8118-366d37a4d1dc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:02.969098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1012/2048] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:02.987270 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c0e69-cb13-43d0-9309-f71091ec94d5 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:02.994895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2e0c0e69-cb13-43d0-9309-f71091ec94d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:03.006039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90bb8a-ae1b-4735-8103-53386a10455d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:03.006812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90bb8a-ae1b-4735-8103-53386a10455d admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:03.006926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90bb8a-ae1b-4735-8103-53386a10455d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:03.007195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90bb8a-ae1b-4735-8103-53386a10455d admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:03.007904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90bb8a-ae1b-4735-8103-53386a10455d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:03.011457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90bb8a-ae1b-4735-8103-53386a10455d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:03.091185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1037/2049] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:02 2022] POST /identity/v3/projects => generated 387 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:03.094008 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c0e69-cb13-43d0-9309-f71091ec94d5 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:03.096334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90bb8a-ae1b-4735-8103-53386a10455d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:03.110363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:03.126182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:03.157325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc0ccc-cc9b-4c51-8a92-1089cac22904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:03.157325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc0ccc-cc9b-4c51-8a92-1089cac22904 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:03.157325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc0ccc-cc9b-4c51-8a92-1089cac22904 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:03.157325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc0ccc-cc9b-4c51-8a92-1089cac22904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:03.201305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90bb8a-ae1b-4735-8103-53386a10455d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:03.207817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f45f4-6338-4de0-b33c-eb98c2fcd53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:03.208396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f45f4-6338-4de0-b33c-eb98c2fcd53a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:03.208676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f45f4-6338-4de0-b33c-eb98c2fcd53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:03.209282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f45f4-6338-4de0-b33c-eb98c2fcd53a admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:03.210100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f45f4-6338-4de0-b33c-eb98c2fcd53a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:03.213223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f45f4-6338-4de0-b33c-eb98c2fcd53a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:03.245690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc0ccc-cc9b-4c51-8a92-1089cac22904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:03.247137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc0ccc-cc9b-4c51-8a92-1089cac22904 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:03.258844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1013/2050] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:03.306951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1038/2051] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:03 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:03.312500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc0ccc-cc9b-4c51-8a92-1089cac22904 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:03.330192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc0ccc-cc9b-4c51-8a92-1089cac22904 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:03.337171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832ddf8-1831-4da3-9912-3233fde11169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:03.337903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ddf8-1831-4da3-9912-3233fde11169 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:03.338206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ddf8-1831-4da3-9912-3233fde11169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:03.338481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ddf8-1831-4da3-9912-3233fde11169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:03.339423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832ddf8-1831-4da3-9912-3233fde11169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:03.341057 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832ddf8-1831-4da3-9912-3233fde11169 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:03.372776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1014/2052] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:03.374539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f45f4-6338-4de0-b33c-eb98c2fcd53a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:03.391767 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f45f4-6338-4de0-b33c-eb98c2fcd53a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:03.397752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65544c61-4be0-4d5b-8e81-5c10b794c898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:03.399431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65544c61-4be0-4d5b-8e81-5c10b794c898 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:03.399918 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65544c61-4be0-4d5b-8e81-5c10b794c898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:03.401194 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65544c61-4be0-4d5b-8e81-5c10b794c898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:03.402131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65544c61-4be0-4d5b-8e81-5c10b794c898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:03.404171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65544c61-4be0-4d5b-8e81-5c10b794c898 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:03.421590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1039/2053] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:03.425356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832ddf8-1831-4da3-9912-3233fde11169 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:03.446989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832ddf8-1831-4da3-9912-3233fde11169 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:03.453182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e55a9-137c-4ecc-8b53-d21db3010ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:03.454030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e55a9-137c-4ecc-8b53-d21db3010ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:03.454311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e55a9-137c-4ecc-8b53-d21db3010ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:03.454640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e55a9-137c-4ecc-8b53-d21db3010ba8 admin admin] PATH_INFO: `/v3/projects/1ade396e1f1a43bf97c93dcb36f4d385/users/e59e46eb11b041c69c2bd241406c1dad/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:03.473573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e55a9-137c-4ecc-8b53-d21db3010ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ade396e1f1a43bf97c93dcb36f4d385, user_id=e59e46eb11b041c69c2bd241406c1dad, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:03.477206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e55a9-137c-4ecc-8b53-d21db3010ba8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:03.548511 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1015/2054] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:03 2022] PUT /identity/v3/projects/1ade396e1f1a43bf97c93dcb36f4d385/users/e59e46eb11b041c69c2bd241406c1dad/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:03.549222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5435be2-eef1-4ebc-ac39-d0aa46761ba8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:03.549514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5435be2-eef1-4ebc-ac39-d0aa46761ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:03.549802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5435be2-eef1-4ebc-ac39-d0aa46761ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:03.606444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c5435be2-eef1-4ebc-ac39-d0aa46761ba8 None None] MFA Rules not processed for user `e59e46eb11b041c69c2bd241406c1dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:03.607789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c5435be2-eef1-4ebc-ac39-d0aa46761ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:03.684704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1040/2055] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:03.689440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e55a9-137c-4ecc-8b53-d21db3010ba8 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:03.755822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e55a9-137c-4ecc-8b53-d21db3010ba8 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:03.761697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ebc66-9cfe-4e66-9351-88e857b74740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:03.762495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ebc66-9cfe-4e66-9351-88e857b74740 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:03.762783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ebc66-9cfe-4e66-9351-88e857b74740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:03.763061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ebc66-9cfe-4e66-9351-88e857b74740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:03.763708 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ebc66-9cfe-4e66-9351-88e857b74740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:03.768114 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ebc66-9cfe-4e66-9351-88e857b74740 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:03.794611 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964d574-9b2f-4c4a-a931-813d36127e2b None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:03.794850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964d574-9b2f-4c4a-a931-813d36127e2b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:03.795130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964d574-9b2f-4c4a-a931-813d36127e2b None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:03.796592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1041/2056] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:39:03.840156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b703b08-8408-417b-887d-df9b772c9a32 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:03.840734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b703b08-8408-417b-887d-df9b772c9a32 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:03.845941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b703b08-8408-417b-887d-df9b772c9a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:03.863271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3b703b08-8408-417b-887d-df9b772c9a32 None None] MFA Rules not processed for user `db1d7e39e8c14f5db40efa743fc3b652`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:03.864617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3b703b08-8408-417b-887d-df9b772c9a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:03.903580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1016/2057] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:03 2022] POST /identity/v3/projects => generated 387 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:03.907870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ebc66-9cfe-4e66-9351-88e857b74740 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:03.939253 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1042/2058] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:39:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:03.947668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:03.952628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b703b08-8408-417b-887d-df9b772c9a32 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:03.954012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3b703b08-8408-417b-887d-df9b772c9a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:03.967442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b703b08-8408-417b-887d-df9b772c9a32 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:03.977958 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:03.994748 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ebc66-9cfe-4e66-9351-88e857b74740 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:04.002414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e2b62-b5ba-4f66-a633-a30d153b84cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:04.003117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e2b62-b5ba-4f66-a633-a30d153b84cd admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:04.003403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e2b62-b5ba-4f66-a633-a30d153b84cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:04.003683 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e2b62-b5ba-4f66-a633-a30d153b84cd admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:04.004378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e2b62-b5ba-4f66-a633-a30d153b84cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:04.010246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e2b62-b5ba-4f66-a633-a30d153b84cd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:04.048680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5eef9-c8b2-4998-a7eb-8fe90adc3ea8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:04.049299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5eef9-c8b2-4998-a7eb-8fe90adc3ea8 service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:04.049602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5eef9-c8b2-4998-a7eb-8fe90adc3ea8 service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:04.049875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5eef9-c8b2-4998-a7eb-8fe90adc3ea8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:04.058934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5eef9-c8b2-4998-a7eb-8fe90adc3ea8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:04.065298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5eef9-c8b2-4998-a7eb-8fe90adc3ea8 service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:04.071480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1043/2059] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:39:04.072421 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5eef9-c8b2-4998-a7eb-8fe90adc3ea8 service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:04.074082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c7b5eef9-c8b2-4998-a7eb-8fe90adc3ea8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:04.087882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5eef9-c8b2-4998-a7eb-8fe90adc3ea8 service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:04.093840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:04.093840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:04.093840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:04.093840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] PATH_INFO: `/v3/users/dfd02222f0b14a588b51a792b02ca852` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:04.097153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfd02222f0b14a588b51a792b02ca852)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:04.099270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:04.129388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1017/2060] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:03 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:04.130383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e2b62-b5ba-4f66-a633-a30d153b84cd admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:04.132033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4c6e2b62-b5ba-4f66-a633-a30d153b84cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:04.144818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e2b62-b5ba-4f66-a633-a30d153b84cd admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:04.149608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:04.157068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:04.157068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:04.157068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] PATH_INFO: `/v3/projects/e13d5f91ab564825bac7c55831d7d027` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:04.157068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e13d5f91ab564825bac7c55831d7d027)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:04.157068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:04.366734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e13d5f91ab564825bac7c55831d7d027', 'request_id': 'req-3a9dc3bd-988d-4529-8303-742fc43ea770'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.367601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfd02222f0b14a588b51a792b02ca852'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.373601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e13d5f91ab564825bac7c55831d7d027', 'request_id': 'req-3a9dc3bd-988d-4529-8303-742fc43ea770'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.454319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfd02222f0b14a588b51a792b02ca852'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.466800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfd02222f0b14a588b51a792b02ca852'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.473955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfd02222f0b14a588b51a792b02ca852'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.474613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfd02222f0b14a588b51a792b02ca852'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.481828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1044/2061] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:04 2022] DELETE /identity/v3/users/dfd02222f0b14a588b51a792b02ca852 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:04.482783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:04.486335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:04.511463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e13d5f91ab564825bac7c55831d7d027', 'request_id': 'req-3a9dc3bd-988d-4529-8303-742fc43ea770'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.530921 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] The token cache is being invalidate because project e13d5f91ab564825bac7c55831d7d027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:04.531198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.533317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.537566 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1018/2062] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:04 2022] DELETE /identity/v3/projects/e13d5f91ab564825bac7c55831d7d027 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:04.539757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:04.541315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:04.568163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad858a9-a5fe-48b2-b55a-825f73465f06 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:04.573280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61f0b6-b969-4d3f-a7fb-10737b666740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:04.578957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61f0b6-b969-4d3f-a7fb-10737b666740 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:04.578957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61f0b6-b969-4d3f-a7fb-10737b666740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:04.578957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61f0b6-b969-4d3f-a7fb-10737b666740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:04.578957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61f0b6-b969-4d3f-a7fb-10737b666740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:04.578957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61f0b6-b969-4d3f-a7fb-10737b666740 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:04.592333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1045/2063] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:04.593579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61f0b6-b969-4d3f-a7fb-10737b666740 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:04.595731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6b61f0b6-b969-4d3f-a7fb-10737b666740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:04.612127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61f0b6-b969-4d3f-a7fb-10737b666740 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:04.618087 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870adb86-ff98-4fda-ab17-8f7729463263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:04.618923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870adb86-ff98-4fda-ab17-8f7729463263 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:04.619249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870adb86-ff98-4fda-ab17-8f7729463263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:04.619524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870adb86-ff98-4fda-ab17-8f7729463263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:04.620258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870adb86-ff98-4fda-ab17-8f7729463263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:04.621068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dc3bd-988d-4529-8303-742fc43ea770 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:04.622302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870adb86-ff98-4fda-ab17-8f7729463263 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:04.626463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d687ad7-de37-49fb-b563-c9e7852d10e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:04.627108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d687ad7-de37-49fb-b563-c9e7852d10e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:04.627390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d687ad7-de37-49fb-b563-c9e7852d10e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:04.627714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d687ad7-de37-49fb-b563-c9e7852d10e1 admin admin] PATH_INFO: `/v3/users/340df16ccb4445deba5fcc0b5d326022` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:04.628727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:04.629285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d687ad7-de37-49fb-b563-c9e7852d10e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=340df16ccb4445deba5fcc0b5d326022)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:04.631400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d687ad7-de37-49fb-b563-c9e7852d10e1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:04.637772 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1046/2064] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:04.641972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870adb86-ff98-4fda-ab17-8f7729463263 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:04.658000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870adb86-ff98-4fda-ab17-8f7729463263 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:04.666187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b310129-39be-4551-8ecd-ac96761f1302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:04.667217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b310129-39be-4551-8ecd-ac96761f1302 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:04.667506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b310129-39be-4551-8ecd-ac96761f1302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:04.667786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b310129-39be-4551-8ecd-ac96761f1302 admin admin] PATH_INFO: `/v3/projects/db384fd64d40479ab4623345a938b101/users/a188179999d340838988242f9d61a874/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:04.686922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b310129-39be-4551-8ecd-ac96761f1302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db384fd64d40479ab4623345a938b101, user_id=a188179999d340838988242f9d61a874, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:04.691741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b310129-39be-4551-8ecd-ac96761f1302 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:04.753918 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1047/2065] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:04 2022] PUT /identity/v3/projects/db384fd64d40479ab4623345a938b101/users/a188179999d340838988242f9d61a874/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:04.759215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b310129-39be-4551-8ecd-ac96761f1302 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:04.759215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8b310129-39be-4551-8ecd-ac96761f1302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:04.808478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=83,keystone:INSERT=4,keystone:DELETE=13 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:04.812545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2d687ad7-de37-49fb-b563-c9e7852d10e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '340df16ccb4445deba5fcc0b5d326022'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.865734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2d687ad7-de37-49fb-b563-c9e7852d10e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '340df16ccb4445deba5fcc0b5d326022'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.874891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b310129-39be-4551-8ecd-ac96761f1302 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:04.879677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2d687ad7-de37-49fb-b563-c9e7852d10e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '340df16ccb4445deba5fcc0b5d326022'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.879991 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:04.890046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2d687ad7-de37-49fb-b563-c9e7852d10e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '340df16ccb4445deba5fcc0b5d326022'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.890222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:04.890222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:04.890408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2d687ad7-de37-49fb-b563-c9e7852d10e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '340df16ccb4445deba5fcc0b5d326022'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:04.890513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] PATH_INFO: `/v3/projects/6cd3d94550bd467f8a88d75ec1208be1` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:04.890513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cd3d94550bd467f8a88d75ec1208be1)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:04.890513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:04.894645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1019/2066] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:04 2022] DELETE /identity/v3/users/340df16ccb4445deba5fcc0b5d326022 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:04.897598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc06a36-98d1-4650-8678-5846c15c55b4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:04.897598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc06a36-98d1-4650-8678-5846c15c55b4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:04.897598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc06a36-98d1-4650-8678-5846c15c55b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:04.926904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2cc06a36-98d1-4650-8678-5846c15c55b4 None None] MFA Rules not processed for user `a188179999d340838988242f9d61a874`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:04.928187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2cc06a36-98d1-4650-8678-5846c15c55b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:05.004206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1020/2067] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:05.008475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cd3d94550bd467f8a88d75ec1208be1', 'request_id': 'req-84b38ebf-3207-4f47-8275-be795b1982a3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:05.009551 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc06a36-98d1-4650-8678-5846c15c55b4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.010987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2cc06a36-98d1-4650-8678-5846c15c55b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:05.015752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cd3d94550bd467f8a88d75ec1208be1', 'request_id': 'req-84b38ebf-3207-4f47-8275-be795b1982a3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:05.077305 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc06a36-98d1-4650-8678-5846c15c55b4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.123251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cd3d94550bd467f8a88d75ec1208be1', 'request_id': 'req-84b38ebf-3207-4f47-8275-be795b1982a3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:05.134743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c6659-26ae-4d5c-b56c-19f268d5d9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.135280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6659-26ae-4d5c-b56c-19f268d5d9dc admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.135563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6659-26ae-4d5c-b56c-19f268d5d9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.135834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6659-26ae-4d5c-b56c-19f268d5d9dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.136492 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c6659-26ae-4d5c-b56c-19f268d5d9dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.138110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] The token cache is being invalidate because project 6cd3d94550bd467f8a88d75ec1208be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:05.138394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c6659-26ae-4d5c-b56c-19f268d5d9dc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.138714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:05.139520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-84b38ebf-3207-4f47-8275-be795b1982a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:05.141601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1048/2068] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:04 2022] DELETE /identity/v3/projects/6cd3d94550bd467f8a88d75ec1208be1 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:05.147294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57963173-7177-4f98-bd4e-6e81ed22874f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.147294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57963173-7177-4f98-bd4e-6e81ed22874f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.147561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57963173-7177-4f98-bd4e-6e81ed22874f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.180398 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-57963173-7177-4f98-bd4e-6e81ed22874f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:05.181028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-57963173-7177-4f98-bd4e-6e81ed22874f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:05.189294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1049/2069] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:05.197342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1448-f730-4cad-8e66-e6496d50bf06 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.197628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1448-f730-4cad-8e66-e6496d50bf06 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.197904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1448-f730-4cad-8e66-e6496d50bf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.217637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e0cf1448-f730-4cad-8e66-e6496d50bf06 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:05.218888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e0cf1448-f730-4cad-8e66-e6496d50bf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:05.228279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1050/2070] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:05.234856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf1448-f730-4cad-8e66-e6496d50bf06 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.245497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1021/2071] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:05 2022] POST /identity/v3/projects => generated 381 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:05.245715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf1448-f730-4cad-8e66-e6496d50bf06 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.249447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c6659-26ae-4d5c-b56c-19f268d5d9dc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.251269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-409c6659-26ae-4d5c-b56c-19f268d5d9dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:05.278537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:05.302310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:05.316889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f943a2-e582-48f2-b06f-620456f0ae97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.317652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f943a2-e582-48f2-b06f-620456f0ae97 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.318604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f943a2-e582-48f2-b06f-620456f0ae97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.318882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f943a2-e582-48f2-b06f-620456f0ae97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.319550 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f943a2-e582-48f2-b06f-620456f0ae97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:05.319849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f943a2-e582-48f2-b06f-620456f0ae97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.322187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f943a2-e582-48f2-b06f-620456f0ae97 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.327722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c6659-26ae-4d5c-b56c-19f268d5d9dc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.332435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d59f8-4c54-42ea-a66f-70821e6ea771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.333036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d59f8-4c54-42ea-a66f-70821e6ea771 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.333320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d59f8-4c54-42ea-a66f-70821e6ea771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.333593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d59f8-4c54-42ea-a66f-70821e6ea771 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.334308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1051/2072] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:39:05.334423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d59f8-4c54-42ea-a66f-70821e6ea771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.336246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d59f8-4c54-42ea-a66f-70821e6ea771 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.338838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f943a2-e582-48f2-b06f-620456f0ae97 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.349551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f943a2-e582-48f2-b06f-620456f0ae97 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.354732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f87051-82fd-42bc-be4f-f9e263f84202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.355354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f87051-82fd-42bc-be4f-f9e263f84202 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.355639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f87051-82fd-42bc-be4f-f9e263f84202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.356034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f87051-82fd-42bc-be4f-f9e263f84202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.356688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f87051-82fd-42bc-be4f-f9e263f84202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.358608 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f87051-82fd-42bc-be4f-f9e263f84202 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.421079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1022/2073] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:05 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:05.422002 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d59f8-4c54-42ea-a66f-70821e6ea771 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.423550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-515d59f8-4c54-42ea-a66f-70821e6ea771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:05.437088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d59f8-4c54-42ea-a66f-70821e6ea771 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.442378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1052/2074] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:05 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:05.442378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f87051-82fd-42bc-be4f-f9e263f84202 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.446653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.447501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.447932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.448430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] PATH_INFO: `/v3/projects/74d82b222d544ba98fb4d4bdfdda8b13` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.451090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d82b222d544ba98fb4d4bdfdda8b13)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.451512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.510396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f87051-82fd-42bc-be4f-f9e263f84202 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.515135 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692a436-a74b-49e4-bf37-c7d50f6d4bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.515737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692a436-a74b-49e4-bf37-c7d50f6d4bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.516039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692a436-a74b-49e4-bf37-c7d50f6d4bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.516315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692a436-a74b-49e4-bf37-c7d50f6d4bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.516969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692a436-a74b-49e4-bf37-c7d50f6d4bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.518654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692a436-a74b-49e4-bf37-c7d50f6d4bf5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.529839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1053/2075] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:05.530872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692a436-a74b-49e4-bf37-c7d50f6d4bf5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.540579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692a436-a74b-49e4-bf37-c7d50f6d4bf5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.545161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b22db-b9f4-49b4-98ca-440cf79266e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.549367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b22db-b9f4-49b4-98ca-440cf79266e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.549367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b22db-b9f4-49b4-98ca-440cf79266e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.549367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b22db-b9f4-49b4-98ca-440cf79266e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.549367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b22db-b9f4-49b4-98ca-440cf79266e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.549367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b22db-b9f4-49b4-98ca-440cf79266e1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.587012 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d82b222d544ba98fb4d4bdfdda8b13', 'request_id': 'req-33bd51fb-0703-4a67-a1b2-bc65c643fda7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:05.593407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d82b222d544ba98fb4d4bdfdda8b13', 'request_id': 'req-33bd51fb-0703-4a67-a1b2-bc65c643fda7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:05.647305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1054/2076] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:05 2022] POST /identity/v3/users => generated 303 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:05.647305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b22db-b9f4-49b4-98ca-440cf79266e1 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.655420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b22db-b9f4-49b4-98ca-440cf79266e1 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.690110 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d82b222d544ba98fb4d4bdfdda8b13', 'request_id': 'req-33bd51fb-0703-4a67-a1b2-bc65c643fda7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:05.702375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] The token cache is being invalidate because project 74d82b222d544ba98fb4d4bdfdda8b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:05.702733 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:05.703496 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:05.708273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1023/2077] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:05 2022] DELETE /identity/v3/projects/74d82b222d544ba98fb4d4bdfdda8b13 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:05.708273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.708273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:05.712184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e5404-37d9-4264-87f5-063f46ac9b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.712831 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e5404-37d9-4264-87f5-063f46ac9b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.713109 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e5404-37d9-4264-87f5-063f46ac9b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.713385 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e5404-37d9-4264-87f5-063f46ac9b7e admin admin] PATH_INFO: `/v3/projects/28914115b3724965b7b4dedc168bee3f/users/d988cbc7cf1a4a2a90bdc5c721edfa05/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.720224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd51fb-0703-4a67-a1b2-bc65c643fda7 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.724931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda02a9-b60a-44d5-a2bf-f74b257096c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.725526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda02a9-b60a-44d5-a2bf-f74b257096c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.727154 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda02a9-b60a-44d5-a2bf-f74b257096c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.727154 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda02a9-b60a-44d5-a2bf-f74b257096c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.727441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda02a9-b60a-44d5-a2bf-f74b257096c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.728849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e5404-37d9-4264-87f5-063f46ac9b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=28914115b3724965b7b4dedc168bee3f, user_id=d988cbc7cf1a4a2a90bdc5c721edfa05, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.729024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda02a9-b60a-44d5-a2bf-f74b257096c6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.732294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e5404-37d9-4264-87f5-063f46ac9b7e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.743439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1024/2078] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:05.744721 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ea808-cd94-4149-a4ee-00dc7397d923 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.745000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ea808-cd94-4149-a4ee-00dc7397d923 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.745287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ea808-cd94-4149-a4ee-00dc7397d923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.764484 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-436ea808-cd94-4149-a4ee-00dc7397d923 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:05.765683 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-436ea808-cd94-4149-a4ee-00dc7397d923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:05.773619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1025/2079] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:05.774406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ea808-cd94-4149-a4ee-00dc7397d923 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.783391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ea808-cd94-4149-a4ee-00dc7397d923 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.787987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf281fa2-4c2d-4b7f-bd37-aab7336a8a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.788577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf281fa2-4c2d-4b7f-bd37-aab7336a8a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.789370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf281fa2-4c2d-4b7f-bd37-aab7336a8a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.789639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf281fa2-4c2d-4b7f-bd37-aab7336a8a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.790730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf281fa2-4c2d-4b7f-bd37-aab7336a8a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.792312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf281fa2-4c2d-4b7f-bd37-aab7336a8a4d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.801751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1026/2080] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:05.802847 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf91d6-a5de-4a58-9970-d9a4d2daae53 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.803126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf91d6-a5de-4a58-9970-d9a4d2daae53 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.803398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf91d6-a5de-4a58-9970-d9a4d2daae53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.820772 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1055/2081] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:05 2022] PUT /identity/v3/projects/28914115b3724965b7b4dedc168bee3f/users/d988cbc7cf1a4a2a90bdc5c721edfa05/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:05.824261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e5404-37d9-4264-87f5-063f46ac9b7e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.830031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-90bf91d6-a5de-4a58-9970-d9a4d2daae53 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:05.831288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-90bf91d6-a5de-4a58-9970-d9a4d2daae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:05.892897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e5404-37d9-4264-87f5-063f46ac9b7e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.896064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:05.902125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1027/2082] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:05.903163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf91d6-a5de-4a58-9970-d9a4d2daae53 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.904615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21be88b-cab3-4052-b5ff-e775a4f00331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.905423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-90bf91d6-a5de-4a58-9970-d9a4d2daae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:05.907548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21be88b-cab3-4052-b5ff-e775a4f00331 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.907808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21be88b-cab3-4052-b5ff-e775a4f00331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.908516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21be88b-cab3-4052-b5ff-e775a4f00331 admin admin] PATH_INFO: `/v3/projects/7f4823f248b64a20b92b5b6da60524b4/users/fabdef76e8c644e8a770c5928dea8ba9/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.921588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:05.921711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf91d6-a5de-4a58-9970-d9a4d2daae53 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.925918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f835f-c2fa-4f98-a6a7-b00b247d783d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.926597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f835f-c2fa-4f98-a6a7-b00b247d783d admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.926879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f835f-c2fa-4f98-a6a7-b00b247d783d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.930130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f835f-c2fa-4f98-a6a7-b00b247d783d admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.930130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f835f-c2fa-4f98-a6a7-b00b247d783d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.930333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f835f-c2fa-4f98-a6a7-b00b247d783d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.935170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21be88b-cab3-4052-b5ff-e775a4f00331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4823f248b64a20b92b5b6da60524b4, user_id=fabdef76e8c644e8a770c5928dea8ba9, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.938018 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21be88b-cab3-4052-b5ff-e775a4f00331 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.949102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1028/2083] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:05.949958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f835f-c2fa-4f98-a6a7-b00b247d783d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:05.960176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f835f-c2fa-4f98-a6a7-b00b247d783d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:05.968980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df60eb-20d2-4186-a4e0-fce75a9a3895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:05.971773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df60eb-20d2-4186-a4e0-fce75a9a3895 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:05.972091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df60eb-20d2-4186-a4e0-fce75a9a3895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:05.972538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df60eb-20d2-4186-a4e0-fce75a9a3895 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:05.973400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df60eb-20d2-4186-a4e0-fce75a9a3895 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:05.973712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df60eb-20d2-4186-a4e0-fce75a9a3895 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:05.975923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df60eb-20d2-4186-a4e0-fce75a9a3895 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:05.990140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1029/2084] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:05.990975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df60eb-20d2-4186-a4e0-fce75a9a3895 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.002071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df60eb-20d2-4186-a4e0-fce75a9a3895 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.005655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1056/2085] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:05 2022] PUT /identity/v3/projects/7f4823f248b64a20b92b5b6da60524b4/users/fabdef76e8c644e8a770c5928dea8ba9/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:06.006060 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21be88b-cab3-4052-b5ff-e775a4f00331 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.073162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee645ff-66ea-4fc1-8e14-956683d9af34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.073825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee645ff-66ea-4fc1-8e14-956683d9af34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.074109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee645ff-66ea-4fc1-8e14-956683d9af34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.074383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee645ff-66ea-4fc1-8e14-956683d9af34 admin admin] PATH_INFO: `/v3/projects/28914115b3724965b7b4dedc168bee3f/users/d988cbc7cf1a4a2a90bdc5c721edfa05/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.076401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee645ff-66ea-4fc1-8e14-956683d9af34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28914115b3724965b7b4dedc168bee3f, user_id=d988cbc7cf1a4a2a90bdc5c721edfa05, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.079075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee645ff-66ea-4fc1-8e14-956683d9af34 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.082682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21be88b-cab3-4052-b5ff-e775a4f00331 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.088118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e82ecb-56e7-4c32-9ff1-edfbbba08a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.088722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e82ecb-56e7-4c32-9ff1-edfbbba08a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.088999 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e82ecb-56e7-4c32-9ff1-edfbbba08a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.089271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e82ecb-56e7-4c32-9ff1-edfbbba08a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.089827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1030/2086] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:05 2022] PUT /identity/v3/projects/28914115b3724965b7b4dedc168bee3f/users/d988cbc7cf1a4a2a90bdc5c721edfa05/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:06.089927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e82ecb-56e7-4c32-9ff1-edfbbba08a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.091172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31340e-67db-4268-9168-fd8be531060d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.091462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31340e-67db-4268-9168-fd8be531060d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.091736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31340e-67db-4268-9168-fd8be531060d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.098258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e82ecb-56e7-4c32-9ff1-edfbbba08a13 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.121324 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7a31340e-67db-4268-9168-fd8be531060d None None] MFA Rules not processed for user `fabdef76e8c644e8a770c5928dea8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:06.122766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7a31340e-67db-4268-9168-fd8be531060d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:06.187064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1031/2087] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:06 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:06.188458 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123883bb-d15f-4d23-a78c-5c4a46659839 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.188828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123883bb-d15f-4d23-a78c-5c4a46659839 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.189112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123883bb-d15f-4d23-a78c-5c4a46659839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.192946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1057/2088] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:06 2022] POST /identity/v3/projects => generated 409 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:06.193986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e82ecb-56e7-4c32-9ff1-edfbbba08a13 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.218715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-123883bb-d15f-4d23-a78c-5c4a46659839 None None] MFA Rules not processed for user `d988cbc7cf1a4a2a90bdc5c721edfa05`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:06.219941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-123883bb-d15f-4d23-a78c-5c4a46659839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:06.260385 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e82ecb-56e7-4c32-9ff1-edfbbba08a13 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.265726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fcdd3-d962-44f9-9825-287d058cff37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.266353 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fcdd3-d962-44f9-9825-287d058cff37 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.268460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fcdd3-d962-44f9-9825-287d058cff37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.268460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fcdd3-d962-44f9-9825-287d058cff37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.268460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fcdd3-d962-44f9-9825-287d058cff37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.269581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fcdd3-d962-44f9-9825-287d058cff37 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.278127 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:06.299171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1032/2089] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:06.299985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123883bb-d15f-4d23-a78c-5c4a46659839 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.309772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123883bb-d15f-4d23-a78c-5c4a46659839 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.314581 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8390f9-1acc-49f5-bc91-3354daa5fd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.315198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8390f9-1acc-49f5-bc91-3354daa5fd7f admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.315483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8390f9-1acc-49f5-bc91-3354daa5fd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.315764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8390f9-1acc-49f5-bc91-3354daa5fd7f admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.316568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8390f9-1acc-49f5-bc91-3354daa5fd7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.318525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8390f9-1acc-49f5-bc91-3354daa5fd7f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.382013 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1058/2090] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:06 2022] POST /identity/v3/projects => generated 403 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:06.382911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fcdd3-d962-44f9-9825-287d058cff37 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.384835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9c5fcdd3-d962-44f9-9825-287d058cff37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:06.396580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1033/2091] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:06 2022] POST /identity/v3/users => generated 305 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:06.396580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8390f9-1acc-49f5-bc91-3354daa5fd7f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.405493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8390f9-1acc-49f5-bc91-3354daa5fd7f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.409797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:06.451402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fcdd3-d962-44f9-9825-287d058cff37 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.463707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127bc97-6cc6-4c7a-8d96-af08e97d8b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.464369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127bc97-6cc6-4c7a-8d96-af08e97d8b54 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.464648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127bc97-6cc6-4c7a-8d96-af08e97d8b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.465012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127bc97-6cc6-4c7a-8d96-af08e97d8b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.478109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b4dcd-2dc5-41e7-8aa4-5ee1cbf33348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.478726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b4dcd-2dc5-41e7-8aa4-5ee1cbf33348 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.479029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b4dcd-2dc5-41e7-8aa4-5ee1cbf33348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.479307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b4dcd-2dc5-41e7-8aa4-5ee1cbf33348 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.479940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b4dcd-2dc5-41e7-8aa4-5ee1cbf33348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.481907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b4dcd-2dc5-41e7-8aa4-5ee1cbf33348 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.535760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127bc97-6cc6-4c7a-8d96-af08e97d8b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.537289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127bc97-6cc6-4c7a-8d96-af08e97d8b54 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.550843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1059/2092] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:06.551661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127bc97-6cc6-4c7a-8d96-af08e97d8b54 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.556499 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1034/2093] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:06 2022] POST /identity/v3/users => generated 308 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:06.560390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b4dcd-2dc5-41e7-8aa4-5ee1cbf33348 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.566783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127bc97-6cc6-4c7a-8d96-af08e97d8b54 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.571590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b4dcd-2dc5-41e7-8aa4-5ee1cbf33348 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.573324 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e39d51-d8ca-476d-8517-b019869c5f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.574189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e39d51-d8ca-476d-8517-b019869c5f4b admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.574620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e39d51-d8ca-476d-8517-b019869c5f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.575207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e39d51-d8ca-476d-8517-b019869c5f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.576085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e39d51-d8ca-476d-8517-b019869c5f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.576417 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6019a0-f180-47bb-9333-99815156ae67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.577033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6019a0-f180-47bb-9333-99815156ae67 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.577307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6019a0-f180-47bb-9333-99815156ae67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.577582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6019a0-f180-47bb-9333-99815156ae67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.578277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6019a0-f180-47bb-9333-99815156ae67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.578890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e39d51-d8ca-476d-8517-b019869c5f4b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.579824 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6019a0-f180-47bb-9333-99815156ae67 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.594544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1035/2094] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:06.595330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1060/2095] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:06.596739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6019a0-f180-47bb-9333-99815156ae67 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.597178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e39d51-d8ca-476d-8517-b019869c5f4b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.610269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6019a0-f180-47bb-9333-99815156ae67 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.610358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e39d51-d8ca-476d-8517-b019869c5f4b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.613940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc04c56-3745-4635-8d23-16ff09b50063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.614088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c139f19-6265-4c40-af41-71bef6ef7bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.614590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc04c56-3745-4635-8d23-16ff09b50063 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.614717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c139f19-6265-4c40-af41-71bef6ef7bef admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.614887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc04c56-3745-4635-8d23-16ff09b50063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.615008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c139f19-6265-4c40-af41-71bef6ef7bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.615144 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc04c56-3745-4635-8d23-16ff09b50063 admin admin] PATH_INFO: `/v3/projects/3beb11c59fdc40bd8490949b13395eda/users/9848030620674475a1e091c4ef73ca37/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.615256 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c139f19-6265-4c40-af41-71bef6ef7bef admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.615876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c139f19-6265-4c40-af41-71bef6ef7bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.618528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c139f19-6265-4c40-af41-71bef6ef7bef admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.635006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1061/2096] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:06.639318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c139f19-6265-4c40-af41-71bef6ef7bef admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.639517 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc04c56-3745-4635-8d23-16ff09b50063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3beb11c59fdc40bd8490949b13395eda, user_id=9848030620674475a1e091c4ef73ca37, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.642650 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc04c56-3745-4635-8d23-16ff09b50063 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.650434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c139f19-6265-4c40-af41-71bef6ef7bef admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.655297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e73436-43f7-496f-bb47-6a4ef9e581b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.655956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e73436-43f7-496f-bb47-6a4ef9e581b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.656287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e73436-43f7-496f-bb47-6a4ef9e581b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.656560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e73436-43f7-496f-bb47-6a4ef9e581b7 admin admin] PATH_INFO: `/v3/projects/efb4cc21e90e4d29a8687dbea3128a2a/users/0ff7e8045a3f4200ad74b20264517c5d/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.676057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e73436-43f7-496f-bb47-6a4ef9e581b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb4cc21e90e4d29a8687dbea3128a2a, user_id=0ff7e8045a3f4200ad74b20264517c5d, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.678673 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e73436-43f7-496f-bb47-6a4ef9e581b7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.712250 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1036/2097] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:06 2022] PUT /identity/v3/projects/3beb11c59fdc40bd8490949b13395eda/users/9848030620674475a1e091c4ef73ca37/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:06.716477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc04c56-3745-4635-8d23-16ff09b50063 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.765359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:06.778594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc04c56-3745-4635-8d23-16ff09b50063 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.783207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d011967-29c1-4878-8c19-55d2976a5c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.783800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d011967-29c1-4878-8c19-55d2976a5c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.784106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d011967-29c1-4878-8c19-55d2976a5c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.784405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d011967-29c1-4878-8c19-55d2976a5c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.785078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d011967-29c1-4878-8c19-55d2976a5c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.786742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d011967-29c1-4878-8c19-55d2976a5c44 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.792639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1062/2098] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:06 2022] PUT /identity/v3/projects/efb4cc21e90e4d29a8687dbea3128a2a/users/0ff7e8045a3f4200ad74b20264517c5d/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:06.796959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1037/2099] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:06.803425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e73436-43f7-496f-bb47-6a4ef9e581b7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.805890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6be452-193f-401b-94f2-dfae59c6705d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.806252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6be452-193f-401b-94f2-dfae59c6705d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.806625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6be452-193f-401b-94f2-dfae59c6705d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.843370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-da6be452-193f-401b-94f2-dfae59c6705d None None] MFA Rules not processed for user `0ff7e8045a3f4200ad74b20264517c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:06.844829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-da6be452-193f-401b-94f2-dfae59c6705d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:06.881785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e73436-43f7-496f-bb47-6a4ef9e581b7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.886618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1196a-8b0f-466b-97bd-b96de6dcf6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:06.887314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1196a-8b0f-466b-97bd-b96de6dcf6d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.887590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1196a-8b0f-466b-97bd-b96de6dcf6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.887860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1196a-8b0f-466b-97bd-b96de6dcf6d9 admin admin] PATH_INFO: `/v3/projects/3beb11c59fdc40bd8490949b13395eda/users/9848030620674475a1e091c4ef73ca37/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.889803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1196a-8b0f-466b-97bd-b96de6dcf6d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3beb11c59fdc40bd8490949b13395eda, user_id=9848030620674475a1e091c4ef73ca37, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:06.892842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1196a-8b0f-466b-97bd-b96de6dcf6d9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:06.904445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1063/2100] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:06 2022] PUT /identity/v3/projects/3beb11c59fdc40bd8490949b13395eda/users/9848030620674475a1e091c4ef73ca37/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:06.914668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1038/2101] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:06 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:06.916863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002e66c-1a67-4b47-9dbd-98fe80f39158 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:06.917156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002e66c-1a67-4b47-9dbd-98fe80f39158 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:06.917430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002e66c-1a67-4b47-9dbd-98fe80f39158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:06.918896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6be452-193f-401b-94f2-dfae59c6705d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:06.932599 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6be452-193f-401b-94f2-dfae59c6705d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:06.965732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4002e66c-1a67-4b47-9dbd-98fe80f39158 None None] MFA Rules not processed for user `9848030620674475a1e091c4ef73ca37`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:06.965732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4002e66c-1a67-4b47-9dbd-98fe80f39158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:06.970385 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:07.014437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf602ef8-ebaf-4621-9433-c8d763f8635b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:07.015279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf602ef8-ebaf-4621-9433-c8d763f8635b admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.016506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf602ef8-ebaf-4621-9433-c8d763f8635b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.017256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf602ef8-ebaf-4621-9433-c8d763f8635b admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.018079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf602ef8-ebaf-4621-9433-c8d763f8635b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:07.020050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf602ef8-ebaf-4621-9433-c8d763f8635b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:07.053912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1064/2102] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:06 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:07.066479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002e66c-1a67-4b47-9dbd-98fe80f39158 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:07.067953 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4002e66c-1a67-4b47-9dbd-98fe80f39158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:07.135767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1039/2103] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:06 2022] POST /identity/v3/projects => generated 407 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:07.137217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002e66c-1a67-4b47-9dbd-98fe80f39158 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:07.137433 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf602ef8-ebaf-4621-9433-c8d763f8635b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:07.195875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07e817-7fd4-4b88-be5b-7375578f2d90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:07.196779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07e817-7fd4-4b88-be5b-7375578f2d90 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.197207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07e817-7fd4-4b88-be5b-7375578f2d90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.197753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07e817-7fd4-4b88-be5b-7375578f2d90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.204927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf602ef8-ebaf-4621-9433-c8d763f8635b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:07.222005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bb02a-c2ee-44e5-b2e5-04cf2d72f81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:07.222787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bb02a-c2ee-44e5-b2e5-04cf2d72f81c admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.222899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bb02a-c2ee-44e5-b2e5-04cf2d72f81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.223368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bb02a-c2ee-44e5-b2e5-04cf2d72f81c admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.224045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bb02a-c2ee-44e5-b2e5-04cf2d72f81c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:07.227208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bb02a-c2ee-44e5-b2e5-04cf2d72f81c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:07.235519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:07.275425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07e817-7fd4-4b88-be5b-7375578f2d90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:07.276982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07e817-7fd4-4b88-be5b-7375578f2d90 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:07.288266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1065/2104] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:07 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:07.311425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1040/2105] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:07 2022] POST /identity/v3/users => generated 310 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:07.314356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07e817-7fd4-4b88-be5b-7375578f2d90 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:07.326024 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07e817-7fd4-4b88-be5b-7375578f2d90 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:07.335667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bf9d7-fafa-4f9a-a072-a6a8a40ff74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:07.335667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bf9d7-fafa-4f9a-a072-a6a8a40ff74a admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.335667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bf9d7-fafa-4f9a-a072-a6a8a40ff74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.335667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bf9d7-fafa-4f9a-a072-a6a8a40ff74a admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.336076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bf9d7-fafa-4f9a-a072-a6a8a40ff74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:07.336076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bf9d7-fafa-4f9a-a072-a6a8a40ff74a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:07.337392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:07.346071 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1066/2106] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:07.350429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bb02a-c2ee-44e5-b2e5-04cf2d72f81c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:07.371526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bb02a-c2ee-44e5-b2e5-04cf2d72f81c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:07.377258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dfc34-6641-40af-b8b3-ab84a0126ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:07.378092 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dfc34-6641-40af-b8b3-ab84a0126ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.378250 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dfc34-6641-40af-b8b3-ab84a0126ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.378560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dfc34-6641-40af-b8b3-ab84a0126ee2 admin admin] PATH_INFO: `/v3/projects/95dbf683755b41a8b8b876a7d2f9e0a2/users/9affde803fd64a18a42341c89b820b7f/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.394668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638dfc34-6641-40af-b8b3-ab84a0126ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95dbf683755b41a8b8b876a7d2f9e0a2, user_id=9affde803fd64a18a42341c89b820b7f, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:07.397487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638dfc34-6641-40af-b8b3-ab84a0126ee2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:07.442468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bf9d7-fafa-4f9a-a072-a6a8a40ff74a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:07.453165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bf9d7-fafa-4f9a-a072-a6a8a40ff74a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:07.457898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:07.458511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.458788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.459217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] PATH_INFO: `/v3/users/fabdef76e8c644e8a770c5928dea8ba9` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.460575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fabdef76e8c644e8a770c5928dea8ba9)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:07.463246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:07.472576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1041/2107] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:07 2022] PUT /identity/v3/projects/95dbf683755b41a8b8b876a7d2f9e0a2/users/9affde803fd64a18a42341c89b820b7f/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:07.477001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dfc34-6641-40af-b8b3-ab84a0126ee2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:07.542538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dfc34-6641-40af-b8b3-ab84a0126ee2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:07.547453 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5603a8-c437-4bd2-b2d4-a71b01d7c737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:07.548267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5603a8-c437-4bd2-b2d4-a71b01d7c737 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.548545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5603a8-c437-4bd2-b2d4-a71b01d7c737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.548948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5603a8-c437-4bd2-b2d4-a71b01d7c737 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.549758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5603a8-c437-4bd2-b2d4-a71b01d7c737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:07.551430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5603a8-c437-4bd2-b2d4-a71b01d7c737 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:07.563708 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1042/2108] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:07.567908 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5603a8-c437-4bd2-b2d4-a71b01d7c737 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:07.578376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5603a8-c437-4bd2-b2d4-a71b01d7c737 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:07.583667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd16755-05a6-4926-9f68-08bf14c1b931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:07.584462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd16755-05a6-4926-9f68-08bf14c1b931 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.584770 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd16755-05a6-4926-9f68-08bf14c1b931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.585026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd16755-05a6-4926-9f68-08bf14c1b931 admin admin] PATH_INFO: `/v3/projects/95dbf683755b41a8b8b876a7d2f9e0a2/users/9affde803fd64a18a42341c89b820b7f/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.587986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd16755-05a6-4926-9f68-08bf14c1b931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95dbf683755b41a8b8b876a7d2f9e0a2, user_id=9affde803fd64a18a42341c89b820b7f, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:07.590570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd16755-05a6-4926-9f68-08bf14c1b931 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:07.610751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1043/2109] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:07 2022] PUT /identity/v3/projects/95dbf683755b41a8b8b876a7d2f9e0a2/users/9affde803fd64a18a42341c89b820b7f/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:07.615449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8aab9-f072-41f6-98c5-59fe6f4d4270 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.615737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8aab9-f072-41f6-98c5-59fe6f4d4270 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.616005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8aab9-f072-41f6-98c5-59fe6f4d4270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.647170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7ad8aab9-f072-41f6-98c5-59fe6f4d4270 None None] MFA Rules not processed for user `9affde803fd64a18a42341c89b820b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:07.648963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7ad8aab9-f072-41f6-98c5-59fe6f4d4270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:07.689131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fabdef76e8c644e8a770c5928dea8ba9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:07.724480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1044/2110] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:07 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:07.736258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8aab9-f072-41f6-98c5-59fe6f4d4270 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:07.763870 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fabdef76e8c644e8a770c5928dea8ba9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:07.771585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:07.777764 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fabdef76e8c644e8a770c5928dea8ba9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:07.784917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fabdef76e8c644e8a770c5928dea8ba9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:07.785720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fabdef76e8c644e8a770c5928dea8ba9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:07.792268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1067/2111] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:07 2022] DELETE /identity/v3/users/fabdef76e8c644e8a770c5928dea8ba9 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:07.793044 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:07.794342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:07.795707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8aab9-f072-41f6-98c5-59fe6f4d4270 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:07.847285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d048d4-ad78-4878-891c-97d166b4258e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:07.847886 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d048d4-ad78-4878-891c-97d166b4258e service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.848219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d048d4-ad78-4878-891c-97d166b4258e service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.848488 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d048d4-ad78-4878-891c-97d166b4258e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.850251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-81d048d4-ad78-4878-891c-97d166b4258e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:07.852278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733f728-fad4-4f56-b8b1-434273fb4e01 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:07.855863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcafcdd-4b42-4fae-ae12-bf0815106470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:07.856502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcafcdd-4b42-4fae-ae12-bf0815106470 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.856787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcafcdd-4b42-4fae-ae12-bf0815106470 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.857061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcafcdd-4b42-4fae-ae12-bf0815106470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.859453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bfcafcdd-4b42-4fae-ae12-bf0815106470 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:07.917671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d048d4-ad78-4878-891c-97d166b4258e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:07.919113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d048d4-ad78-4878-891c-97d166b4258e service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:07.922863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcafcdd-4b42-4fae-ae12-bf0815106470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:07.924331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcafcdd-4b42-4fae-ae12-bf0815106470 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:07.929486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1045/2112] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:07 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:39:07.930551 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eb7b3-5f0f-42f6-9859-7daf0d2fafaf None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:07.930932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eb7b3-5f0f-42f6-9859-7daf0d2fafaf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:07.931215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eb7b3-5f0f-42f6-9859-7daf0d2fafaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:07.933639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1068/2113] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:07.951148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ea4eb7b3-5f0f-42f6-9859-7daf0d2fafaf None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:07.952331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ea4eb7b3-5f0f-42f6-9859-7daf0d2fafaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:08.038033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1046/2114] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:08.056195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcafcdd-4b42-4fae-ae12-bf0815106470 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:08.056195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bfcafcdd-4b42-4fae-ae12-bf0815106470 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:08.084326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:08.113618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcafcdd-4b42-4fae-ae12-bf0815106470 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:08.117343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fce790-9ce6-48c1-b9c4-efbf43e3d475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:08.117978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fce790-9ce6-48c1-b9c4-efbf43e3d475 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:08.118262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fce790-9ce6-48c1-b9c4-efbf43e3d475 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:08.118533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fce790-9ce6-48c1-b9c4-efbf43e3d475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:08.127429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fce790-9ce6-48c1-b9c4-efbf43e3d475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:08.128917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fce790-9ce6-48c1-b9c4-efbf43e3d475 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:08.139965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1069/2115] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:08.413474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4eb7b3-5f0f-42f6-9859-7daf0d2fafaf None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:08.416152 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ea4eb7b3-5f0f-42f6-9859-7daf0d2fafaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:08.428143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4eb7b3-5f0f-42f6-9859-7daf0d2fafaf None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:08.432410 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a52e9-ff63-4ced-986a-9809838e047b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:08.436163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a52e9-ff63-4ced-986a-9809838e047b service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:08.436163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a52e9-ff63-4ced-986a-9809838e047b service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:08.436163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a52e9-ff63-4ced-986a-9809838e047b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:08.436163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b55a52e9-ff63-4ced-986a-9809838e047b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:08.516066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:08.538370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a52e9-ff63-4ced-986a-9809838e047b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:08.540070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a52e9-ff63-4ced-986a-9809838e047b service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:08.549934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1047/2116] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:08 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:08.636041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fce790-9ce6-48c1-b9c4-efbf43e3d475 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:08.637966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-36fce790-9ce6-48c1-b9c4-efbf43e3d475 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:08.658442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fce790-9ce6-48c1-b9c4-efbf43e3d475 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:08.667467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:08.668167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:08.668572 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:08.668963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] PATH_INFO: `/v3/projects/7f4823f248b64a20b92b5b6da60524b4` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:08.670371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f4823f248b64a20b92b5b6da60524b4)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:08.675670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:08.798836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f4823f248b64a20b92b5b6da60524b4', 'request_id': 'req-c0e38626-57c7-4433-8f98-7cd9d84e18d0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:08.807678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f4823f248b64a20b92b5b6da60524b4', 'request_id': 'req-c0e38626-57c7-4433-8f98-7cd9d84e18d0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:08.912123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f4823f248b64a20b92b5b6da60524b4', 'request_id': 'req-c0e38626-57c7-4433-8f98-7cd9d84e18d0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:08.923800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] The token cache is being invalidate because project 7f4823f248b64a20b92b5b6da60524b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:08.924127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:08.925817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:08.927326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1070/2117] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:08 2022] DELETE /identity/v3/projects/7f4823f248b64a20b92b5b6da60524b4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:08.932005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a52e9-ff63-4ced-986a-9809838e047b service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:08.943902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a52e9-ff63-4ced-986a-9809838e047b service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:09.009307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705dfdf-4a7a-4e59-9831-7a4d9895fd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:09.009925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705dfdf-4a7a-4e59-9831-7a4d9895fd5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:09.010210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705dfdf-4a7a-4e59-9831-7a4d9895fd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:09.010483 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705dfdf-4a7a-4e59-9831-7a4d9895fd5e admin admin] PATH_INFO: `/v3/users/9848030620674475a1e091c4ef73ca37` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:09.011614 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705dfdf-4a7a-4e59-9831-7a4d9895fd5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9848030620674475a1e091c4ef73ca37)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:09.013553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705dfdf-4a7a-4e59-9831-7a4d9895fd5e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:09.245404 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4705dfdf-4a7a-4e59-9831-7a4d9895fd5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9848030620674475a1e091c4ef73ca37'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:09.328564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4705dfdf-4a7a-4e59-9831-7a4d9895fd5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9848030620674475a1e091c4ef73ca37'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:09.342529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4705dfdf-4a7a-4e59-9831-7a4d9895fd5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9848030620674475a1e091c4ef73ca37'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:09.343210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4705dfdf-4a7a-4e59-9831-7a4d9895fd5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9848030620674475a1e091c4ef73ca37'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:09.343907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4705dfdf-4a7a-4e59-9831-7a4d9895fd5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9848030620674475a1e091c4ef73ca37'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:09.357040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1048/2118] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:08 2022] DELETE /identity/v3/users/9848030620674475a1e091c4ef73ca37 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:09.810986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:09.812486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:09.825588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e38626-57c7-4433-8f98-7cd9d84e18d0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:09.895363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:09.896055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:09.896469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:09.896752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] PATH_INFO: `/v3/projects/3beb11c59fdc40bd8490949b13395eda` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:09.897919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3beb11c59fdc40bd8490949b13395eda)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:09.900532 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:10.010519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3beb11c59fdc40bd8490949b13395eda', 'request_id': 'req-65331dd6-8109-49c1-9127-3f7afd2dbaca'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:10.012459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:10.017255 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3beb11c59fdc40bd8490949b13395eda', 'request_id': 'req-65331dd6-8109-49c1-9127-3f7afd2dbaca'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:10.095397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3beb11c59fdc40bd8490949b13395eda', 'request_id': 'req-65331dd6-8109-49c1-9127-3f7afd2dbaca'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:10.108197 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] The token cache is being invalidate because project 3beb11c59fdc40bd8490949b13395eda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:10.108496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:10.109200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-65331dd6-8109-49c1-9127-3f7afd2dbaca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:10.114389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1071/2119] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:09 2022] DELETE /identity/v3/projects/3beb11c59fdc40bd8490949b13395eda => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:10.116370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30411a7-152c-481d-8d6b-7010224ab104 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:10.116993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30411a7-152c-481d-8d6b-7010224ab104 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:10.117321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30411a7-152c-481d-8d6b-7010224ab104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:10.136989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c30411a7-152c-481d-8d6b-7010224ab104 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:10.138201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c30411a7-152c-481d-8d6b-7010224ab104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:10.191879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:10.201947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1049/2120] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:10.209581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32d7c9-88d8-4bd2-a2d1-613213bb65a5 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:10.209867 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32d7c9-88d8-4bd2-a2d1-613213bb65a5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:10.210206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32d7c9-88d8-4bd2-a2d1-613213bb65a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:10.230403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3d32d7c9-88d8-4bd2-a2d1-613213bb65a5 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:10.231619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3d32d7c9-88d8-4bd2-a2d1-613213bb65a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:10.239335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1072/2121] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:10.243756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30411a7-152c-481d-8d6b-7010224ab104 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:10.253286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30411a7-152c-481d-8d6b-7010224ab104 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:10.258600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39241e-2943-40e0-b580-b1576c955eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:10.259191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39241e-2943-40e0-b580-b1576c955eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:10.259467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39241e-2943-40e0-b580-b1576c955eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:10.265894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39241e-2943-40e0-b580-b1576c955eb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:10.265894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39241e-2943-40e0-b580-b1576c955eb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:10.265894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39241e-2943-40e0-b580-b1576c955eb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:10.265894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39241e-2943-40e0-b580-b1576c955eb6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:10.274023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1050/2122] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:10.278268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32d7c9-88d8-4bd2-a2d1-613213bb65a5 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:10.287556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32d7c9-88d8-4bd2-a2d1-613213bb65a5 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:10.292369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f1acf-0ddb-49d4-8a29-8ed29791ed8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:10.292956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f1acf-0ddb-49d4-8a29-8ed29791ed8e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:10.294593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f1acf-0ddb-49d4-8a29-8ed29791ed8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:10.294593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f1acf-0ddb-49d4-8a29-8ed29791ed8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:10.294593 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f1acf-0ddb-49d4-8a29-8ed29791ed8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:10.295948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f1acf-0ddb-49d4-8a29-8ed29791ed8e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:10.390170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1073/2123] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:10 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:10.399622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39241e-2943-40e0-b580-b1576c955eb6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:10.405363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39241e-2943-40e0-b580-b1576c955eb6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:10.464700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f33398-a277-4364-8845-c1df48d43a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:10.465332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f33398-a277-4364-8845-c1df48d43a77 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:10.465613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f33398-a277-4364-8845-c1df48d43a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:10.465884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f33398-a277-4364-8845-c1df48d43a77 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:10.466514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f33398-a277-4364-8845-c1df48d43a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:10.468457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f33398-a277-4364-8845-c1df48d43a77 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:10.563774 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1051/2124] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:10 2022] POST /identity/v3/users => generated 283 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:10.567804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f1acf-0ddb-49d4-8a29-8ed29791ed8e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:10.578201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f1acf-0ddb-49d4-8a29-8ed29791ed8e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:10.583195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f4628-ecfb-45f0-8384-2e979032ea29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:10.583818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4628-ecfb-45f0-8384-2e979032ea29 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:10.584129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4628-ecfb-45f0-8384-2e979032ea29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:10.584443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4628-ecfb-45f0-8384-2e979032ea29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:10.585089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f4628-ecfb-45f0-8384-2e979032ea29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:10.587566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f4628-ecfb-45f0-8384-2e979032ea29 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:10.598836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1074/2125] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:10.603856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f33398-a277-4364-8845-c1df48d43a77 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:10.614537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f33398-a277-4364-8845-c1df48d43a77 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:10.623081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644766f-733e-4123-aed9-5719bb84e3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:10.623081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644766f-733e-4123-aed9-5719bb84e3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:10.623081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644766f-733e-4123-aed9-5719bb84e3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:10.623081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644766f-733e-4123-aed9-5719bb84e3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:10.623437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644766f-733e-4123-aed9-5719bb84e3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:10.623437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644766f-733e-4123-aed9-5719bb84e3cf admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:10.634909 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1052/2126] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:10.639219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f4628-ecfb-45f0-8384-2e979032ea29 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:10.649498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f4628-ecfb-45f0-8384-2e979032ea29 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:10.654300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc463037-b9fd-4fe3-90bb-3db63a54991a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:10.654945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc463037-b9fd-4fe3-90bb-3db63a54991a admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:10.655222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc463037-b9fd-4fe3-90bb-3db63a54991a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:10.655490 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc463037-b9fd-4fe3-90bb-3db63a54991a admin admin] PATH_INFO: `/v3/projects/74a91dffb5724143b789b2c8bcf8ef29/users/ba72ebd0c7304c8d94f9f1aec7a022cc/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:10.670145 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc463037-b9fd-4fe3-90bb-3db63a54991a admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a91dffb5724143b789b2c8bcf8ef29, user_id=ba72ebd0c7304c8d94f9f1aec7a022cc, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:10.672839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc463037-b9fd-4fe3-90bb-3db63a54991a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:10.795841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1075/2127] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:10 2022] PUT /identity/v3/projects/74a91dffb5724143b789b2c8bcf8ef29/users/ba72ebd0c7304c8d94f9f1aec7a022cc/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:10.806237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9f58d-50b8-464f-91a9-13532464a857 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:10.806523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9f58d-50b8-464f-91a9-13532464a857 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:10.806799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9f58d-50b8-464f-91a9-13532464a857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:10.837643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-61f9f58d-50b8-464f-91a9-13532464a857 None None] MFA Rules not processed for user `ba72ebd0c7304c8d94f9f1aec7a022cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:10.838900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-61f9f58d-50b8-464f-91a9-13532464a857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:10.912033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1053/2128] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:10.923432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc463037-b9fd-4fe3-90bb-3db63a54991a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:10.985049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc463037-b9fd-4fe3-90bb-3db63a54991a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:10.989676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5557c-aff4-439d-a2f1-ed3e1f59286d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:10.990273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5557c-aff4-439d-a2f1-ed3e1f59286d admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:10.990551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5557c-aff4-439d-a2f1-ed3e1f59286d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:10.990821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5557c-aff4-439d-a2f1-ed3e1f59286d admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:10.991447 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5557c-aff4-439d-a2f1-ed3e1f59286d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:10.993393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5557c-aff4-439d-a2f1-ed3e1f59286d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:11.131280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1076/2129] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:10 2022] POST /identity/v3/projects => generated 365 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:11.136101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9f58d-50b8-464f-91a9-13532464a857 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:11.146661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9f58d-50b8-464f-91a9-13532464a857 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:11.196508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:11.205083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c783f-cfb4-47ae-93db-ab55d28feebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:11.205762 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c783f-cfb4-47ae-93db-ab55d28feebc admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:11.206040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c783f-cfb4-47ae-93db-ab55d28feebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:11.206308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c783f-cfb4-47ae-93db-ab55d28feebc admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:11.206926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c783f-cfb4-47ae-93db-ab55d28feebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:11.208898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c783f-cfb4-47ae-93db-ab55d28feebc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:11.288718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1054/2130] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:11 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:11.292704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5557c-aff4-439d-a2f1-ed3e1f59286d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:11.303144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5557c-aff4-439d-a2f1-ed3e1f59286d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:11.308408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0e210-7647-4dbe-aa8a-8424f4e9a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:11.309008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0e210-7647-4dbe-aa8a-8424f4e9a2df admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:11.309321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0e210-7647-4dbe-aa8a-8424f4e9a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:11.309595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0e210-7647-4dbe-aa8a-8424f4e9a2df admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:11.310250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0e210-7647-4dbe-aa8a-8424f4e9a2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:11.311843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0e210-7647-4dbe-aa8a-8424f4e9a2df admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:11.323329 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1077/2131] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:11.327403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c783f-cfb4-47ae-93db-ab55d28feebc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:11.337391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c783f-cfb4-47ae-93db-ab55d28feebc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:11.342218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e2823-29c3-4cba-8b10-6658e51ea0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:11.342748 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e2823-29c3-4cba-8b10-6658e51ea0a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:11.343056 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e2823-29c3-4cba-8b10-6658e51ea0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:11.343357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e2823-29c3-4cba-8b10-6658e51ea0a3 admin admin] PATH_INFO: `/v3/projects/cedba8b3f0d44b62ad243bfb874415cf/users/84553f86982d4b05949051d5a5a0004c/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:11.359027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e2823-29c3-4cba-8b10-6658e51ea0a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cedba8b3f0d44b62ad243bfb874415cf, user_id=84553f86982d4b05949051d5a5a0004c, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:11.361728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e2823-29c3-4cba-8b10-6658e51ea0a3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:11.432452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1055/2132] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:11 2022] PUT /identity/v3/projects/cedba8b3f0d44b62ad243bfb874415cf/users/84553f86982d4b05949051d5a5a0004c/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:11.436516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0e210-7647-4dbe-aa8a-8424f4e9a2df admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:11.447839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0e210-7647-4dbe-aa8a-8424f4e9a2df admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:11.493086 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:11.502413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1af688-beeb-4ee6-bdf1-8e17e5096484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:11.503010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af688-beeb-4ee6-bdf1-8e17e5096484 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:11.503287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af688-beeb-4ee6-bdf1-8e17e5096484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:11.503565 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af688-beeb-4ee6-bdf1-8e17e5096484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:11.504267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1af688-beeb-4ee6-bdf1-8e17e5096484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:11.505824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1af688-beeb-4ee6-bdf1-8e17e5096484 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:11.516290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1078/2133] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:11.520553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e2823-29c3-4cba-8b10-6658e51ea0a3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:11.531240 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e2823-29c3-4cba-8b10-6658e51ea0a3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:11.536263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b621b5-1755-4e86-8bbe-b261584ec909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:11.536917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b621b5-1755-4e86-8bbe-b261584ec909 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:11.537196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b621b5-1755-4e86-8bbe-b261584ec909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:11.537469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b621b5-1755-4e86-8bbe-b261584ec909 admin admin] PATH_INFO: `/v3/projects/cedba8b3f0d44b62ad243bfb874415cf/users/84553f86982d4b05949051d5a5a0004c/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:11.540294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b621b5-1755-4e86-8bbe-b261584ec909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cedba8b3f0d44b62ad243bfb874415cf, user_id=84553f86982d4b05949051d5a5a0004c, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:11.544528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b621b5-1755-4e86-8bbe-b261584ec909 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:11.555488 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1056/2134] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:11 2022] PUT /identity/v3/projects/cedba8b3f0d44b62ad243bfb874415cf/users/84553f86982d4b05949051d5a5a0004c/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:11.566945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c811b4c-d62f-4c73-b377-a6d946605c6d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:11.567232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c811b4c-d62f-4c73-b377-a6d946605c6d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:11.567509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c811b4c-d62f-4c73-b377-a6d946605c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:11.599559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7c811b4c-d62f-4c73-b377-a6d946605c6d None None] MFA Rules not processed for user `84553f86982d4b05949051d5a5a0004c`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:11.600802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7c811b4c-d62f-4c73-b377-a6d946605c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:11.676802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1079/2135] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:11.689655 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b621b5-1755-4e86-8bbe-b261584ec909 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:11.691506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-59b621b5-1755-4e86-8bbe-b261584ec909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:11.752594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b621b5-1755-4e86-8bbe-b261584ec909 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:11.756381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a14fc-3cc7-4cc9-9b55-8cdf68524dfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:11.757099 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a14fc-3cc7-4cc9-9b55-8cdf68524dfd service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:11.757697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a14fc-3cc7-4cc9-9b55-8cdf68524dfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:11.758014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a14fc-3cc7-4cc9-9b55-8cdf68524dfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:11.770615 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a14fc-3cc7-4cc9-9b55-8cdf68524dfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:11.772077 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a14fc-3cc7-4cc9-9b55-8cdf68524dfd service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:11.782998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1057/2136] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:11.986910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c811b4c-d62f-4c73-b377-a6d946605c6d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:11.997645 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c811b4c-d62f-4c73-b377-a6d946605c6d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:12.001904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e4700-78ed-4c9d-9160-1292057b3f69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:12.002506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e4700-78ed-4c9d-9160-1292057b3f69 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:12.002783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e4700-78ed-4c9d-9160-1292057b3f69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:12.003054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e4700-78ed-4c9d-9160-1292057b3f69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:12.061501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e4700-78ed-4c9d-9160-1292057b3f69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:12.062943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e4700-78ed-4c9d-9160-1292057b3f69 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:12.072213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1080/2137] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:16.156280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a14fc-3cc7-4cc9-9b55-8cdf68524dfd service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:16.157674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cd3a14fc-3cc7-4cc9-9b55-8cdf68524dfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:16.170014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a14fc-3cc7-4cc9-9b55-8cdf68524dfd service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:16.174004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89614a08-cbb4-4e03-84a6-fcfeca9c40db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:16.174617 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89614a08-cbb4-4e03-84a6-fcfeca9c40db service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:16.174892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89614a08-cbb4-4e03-84a6-fcfeca9c40db service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:16.175158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89614a08-cbb4-4e03-84a6-fcfeca9c40db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:16.176893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-89614a08-cbb4-4e03-84a6-fcfeca9c40db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:16.239009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89614a08-cbb4-4e03-84a6-fcfeca9c40db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:16.240478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89614a08-cbb4-4e03-84a6-fcfeca9c40db service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:16.250678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1058/2138] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:17.768120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e4700-78ed-4c9d-9160-1292057b3f69 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:17.839831 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e4700-78ed-4c9d-9160-1292057b3f69 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:17.845043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:17.845283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:17.845566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:17.845845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] PATH_INFO: `/v3/users/ba72ebd0c7304c8d94f9f1aec7a022cc` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:17.847469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba72ebd0c7304c8d94f9f1aec7a022cc)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:17.849469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:17.865067 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:18.051379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba72ebd0c7304c8d94f9f1aec7a022cc'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:18.117181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba72ebd0c7304c8d94f9f1aec7a022cc'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:18.130272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba72ebd0c7304c8d94f9f1aec7a022cc'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:18.138254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba72ebd0c7304c8d94f9f1aec7a022cc'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:18.138905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba72ebd0c7304c8d94f9f1aec7a022cc'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:18.145032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1081/2139] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:17 2022] DELETE /identity/v3/users/ba72ebd0c7304c8d94f9f1aec7a022cc => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:18.149960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fdfd0-83fb-46e6-b4ba-58a6afebf9e2 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:18.150443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fdfd0-83fb-46e6-b4ba-58a6afebf9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:18.150533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fdfd0-83fb-46e6-b4ba-58a6afebf9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:18.169609 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-755fdfd0-83fb-46e6-b4ba-58a6afebf9e2 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:18.170898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-755fdfd0-83fb-46e6-b4ba-58a6afebf9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:18.212969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:18.229382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1059/2140] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:18.246706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:18.248137 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:18.305916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1713cb-672d-4b02-88b0-4e97166bcd8b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:18.309739 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10d98c-af45-4531-98e7-1dabc1e6caf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:18.310383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10d98c-af45-4531-98e7-1dabc1e6caf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:18.310660 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10d98c-af45-4531-98e7-1dabc1e6caf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:18.310930 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10d98c-af45-4531-98e7-1dabc1e6caf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:18.319996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10d98c-af45-4531-98e7-1dabc1e6caf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:18.322055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10d98c-af45-4531-98e7-1dabc1e6caf8 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:18.331477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1082/2141] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:18.754038 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fdfd0-83fb-46e6-b4ba-58a6afebf9e2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:18.755454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-755fdfd0-83fb-46e6-b4ba-58a6afebf9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:18.792968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fdfd0-83fb-46e6-b4ba-58a6afebf9e2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:18.797643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:18.798280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:18.798557 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:18.798830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] PATH_INFO: `/v3/projects/74a91dffb5724143b789b2c8bcf8ef29` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:18.800055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a91dffb5724143b789b2c8bcf8ef29)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:18.802081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:18.927944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a91dffb5724143b789b2c8bcf8ef29', 'request_id': 'req-1630460c-1353-4c1c-b4a6-4e615c736ca2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:18.935044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a91dffb5724143b789b2c8bcf8ef29', 'request_id': 'req-1630460c-1353-4c1c-b4a6-4e615c736ca2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.003408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a91dffb5724143b789b2c8bcf8ef29', 'request_id': 'req-1630460c-1353-4c1c-b4a6-4e615c736ca2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.020577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] The token cache is being invalidate because project 74a91dffb5724143b789b2c8bcf8ef29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:19.020875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.021764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.028360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1060/2142] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:18 2022] DELETE /identity/v3/projects/74a91dffb5724143b789b2c8bcf8ef29 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:19.029651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10d98c-af45-4531-98e7-1dabc1e6caf8 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:19.043322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10d98c-af45-4531-98e7-1dabc1e6caf8 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:19.049480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:19.051501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:19.121467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:19.130594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:19.130594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:19.130594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] PATH_INFO: `/v3/users/84553f86982d4b05949051d5a5a0004c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:19.130594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] RBAC: Authorizing `identity:delete_user(user_id=84553f86982d4b05949051d5a5a0004c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:19.130594 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:19.131615 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630460c-1353-4c1c-b4a6-4e615c736ca2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:19.151855 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f59b587-eefe-4fa7-9d2a-1ed15356f063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:19.152655 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59b587-eefe-4fa7-9d2a-1ed15356f063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:19.152999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59b587-eefe-4fa7-9d2a-1ed15356f063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:19.153344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59b587-eefe-4fa7-9d2a-1ed15356f063 admin admin] PATH_INFO: `/v3/users/0ff7e8045a3f4200ad74b20264517c5d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:19.154641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f59b587-eefe-4fa7-9d2a-1ed15356f063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ff7e8045a3f4200ad74b20264517c5d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:19.156786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f59b587-eefe-4fa7-9d2a-1ed15356f063 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:19.305824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:19.315873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8f59b587-eefe-4fa7-9d2a-1ed15356f063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ff7e8045a3f4200ad74b20264517c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.318072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84553f86982d4b05949051d5a5a0004c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.391806 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8f59b587-eefe-4fa7-9d2a-1ed15356f063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ff7e8045a3f4200ad74b20264517c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.406439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8f59b587-eefe-4fa7-9d2a-1ed15356f063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ff7e8045a3f4200ad74b20264517c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.407304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8f59b587-eefe-4fa7-9d2a-1ed15356f063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ff7e8045a3f4200ad74b20264517c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.407831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8f59b587-eefe-4fa7-9d2a-1ed15356f063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ff7e8045a3f4200ad74b20264517c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.409086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:19.421508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1061/2143] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:19 2022] DELETE /identity/v3/users/0ff7e8045a3f4200ad74b20264517c5d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:19.425866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522f18e-5a93-4c70-b4fd-9302d77ae1dc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:19.426157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522f18e-5a93-4c70-b4fd-9302d77ae1dc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:19.426439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522f18e-5a93-4c70-b4fd-9302d77ae1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:19.436971 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84553f86982d4b05949051d5a5a0004c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.445487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4522f18e-5a93-4c70-b4fd-9302d77ae1dc None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:19.447187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4522f18e-5a93-4c70-b4fd-9302d77ae1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:19.450889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84553f86982d4b05949051d5a5a0004c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.458468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84553f86982d4b05949051d5a5a0004c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.459075 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84553f86982d4b05949051d5a5a0004c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:19.465755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1083/2144] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:19 2022] DELETE /identity/v3/users/84553f86982d4b05949051d5a5a0004c => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:19.515188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1062/2145] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:19.528982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:19.530821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:19.591008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a5c14-1ec7-411b-91f1-035ebf4af78a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:19.594741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ea058-d234-4627-bbf3-cec4ba650c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:19.595406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ea058-d234-4627-bbf3-cec4ba650c05 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:19.595690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ea058-d234-4627-bbf3-cec4ba650c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:19.596146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ea058-d234-4627-bbf3-cec4ba650c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:19.664281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ea058-d234-4627-bbf3-cec4ba650c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:19.666190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ea058-d234-4627-bbf3-cec4ba650c05 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:19.678554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1084/2146] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:39:19.971928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522f18e-5a93-4c70-b4fd-9302d77ae1dc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:19.976991 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4522f18e-5a93-4c70-b4fd-9302d77ae1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:19.993894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522f18e-5a93-4c70-b4fd-9302d77ae1dc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:19.999259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:20.000241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:20.000369 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:20.000856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] PATH_INFO: `/v3/projects/cedba8b3f0d44b62ad243bfb874415cf` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:20.002349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cedba8b3f0d44b62ad243bfb874415cf)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:20.004804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:20.169390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cedba8b3f0d44b62ad243bfb874415cf', 'request_id': 'req-69e2778e-63b3-4c32-97df-1d6e69b64d25'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.176119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cedba8b3f0d44b62ad243bfb874415cf', 'request_id': 'req-69e2778e-63b3-4c32-97df-1d6e69b64d25'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.261224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cedba8b3f0d44b62ad243bfb874415cf', 'request_id': 'req-69e2778e-63b3-4c32-97df-1d6e69b64d25'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.270085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ea058-d234-4627-bbf3-cec4ba650c05 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:20.271674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-207ea058-d234-4627-bbf3-cec4ba650c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:20.278526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] The token cache is being invalidate because project cedba8b3f0d44b62ad243bfb874415cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:20.278822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.281221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-69e2778e-63b3-4c32-97df-1d6e69b64d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.296461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1063/2147] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:19 2022] DELETE /identity/v3/projects/cedba8b3f0d44b62ad243bfb874415cf => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:20.296461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3dd4c-3061-4f58-b716-fb255dc63e89 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:20.296461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3dd4c-3061-4f58-b716-fb255dc63e89 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:20.296461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3dd4c-3061-4f58-b716-fb255dc63e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:20.296955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ea058-d234-4627-bbf3-cec4ba650c05 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:20.319701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-21c3dd4c-3061-4f58-b716-fb255dc63e89 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:20.321411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-21c3dd4c-3061-4f58-b716-fb255dc63e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:20.361071 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:20.361698 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:20.362271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:20.362546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] PATH_INFO: `/v3/projects/efb4cc21e90e4d29a8687dbea3128a2a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:20.363827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb4cc21e90e4d29a8687dbea3128a2a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:20.365804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:20.369016 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:20.392074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1064/2148] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:20.400479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32a5ff-6c55-4b0f-a3f7-30b5f0b36ea5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:20.400764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32a5ff-6c55-4b0f-a3f7-30b5f0b36ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:20.401224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32a5ff-6c55-4b0f-a3f7-30b5f0b36ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:20.423044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1d32a5ff-6c55-4b0f-a3f7-30b5f0b36ea5 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:20.424304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1d32a5ff-6c55-4b0f-a3f7-30b5f0b36ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:20.432701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1065/2149] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:20.446890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32a5ff-6c55-4b0f-a3f7-30b5f0b36ea5 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:20.456587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32a5ff-6c55-4b0f-a3f7-30b5f0b36ea5 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:20.462616 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b69433-d41e-4570-ab94-7084471265a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:20.463226 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69433-d41e-4570-ab94-7084471265a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:20.475242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69433-d41e-4570-ab94-7084471265a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:20.475242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69433-d41e-4570-ab94-7084471265a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:20.475242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b69433-d41e-4570-ab94-7084471265a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:20.475242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b69433-d41e-4570-ab94-7084471265a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:20.475242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b69433-d41e-4570-ab94-7084471265a0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:20.475242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:20.480279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1066/2150] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:20.484041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb4cc21e90e4d29a8687dbea3128a2a', 'request_id': 'req-bcf71674-439f-4636-aa37-3812e62343ec'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.484573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b69433-d41e-4570-ab94-7084471265a0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:20.491071 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb4cc21e90e4d29a8687dbea3128a2a', 'request_id': 'req-bcf71674-439f-4636-aa37-3812e62343ec'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.495016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b69433-d41e-4570-ab94-7084471265a0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:20.552104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149745ea-7b09-4f01-80cb-df8aa1521d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:20.552708 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149745ea-7b09-4f01-80cb-df8aa1521d7e admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:20.552987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149745ea-7b09-4f01-80cb-df8aa1521d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:20.553261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149745ea-7b09-4f01-80cb-df8aa1521d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:20.553879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149745ea-7b09-4f01-80cb-df8aa1521d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:20.556085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149745ea-7b09-4f01-80cb-df8aa1521d7e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:20.597656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb4cc21e90e4d29a8687dbea3128a2a', 'request_id': 'req-bcf71674-439f-4636-aa37-3812e62343ec'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.611843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] The token cache is being invalidate because project efb4cc21e90e4d29a8687dbea3128a2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:20.612368 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.613514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.618513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1085/2151] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:20 2022] DELETE /identity/v3/projects/efb4cc21e90e4d29a8687dbea3128a2a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:20.619457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:20.621467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:20.630230 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1067/2152] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:20 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:20.634295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149745ea-7b09-4f01-80cb-df8aa1521d7e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:20.637186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-149745ea-7b09-4f01-80cb-df8aa1521d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:20.639837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf71674-439f-4636-aa37-3812e62343ec admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:20.712839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:20.713564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:20.713920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:20.714284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] PATH_INFO: `/v3/users/9affde803fd64a18a42341c89b820b7f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:20.715727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9affde803fd64a18a42341c89b820b7f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:20.717854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:20.720563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149745ea-7b09-4f01-80cb-df8aa1521d7e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:20.726229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45620359-44d3-4a60-80cd-a375a748ef6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:20.727491 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45620359-44d3-4a60-80cd-a375a748ef6e admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:20.728136 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45620359-44d3-4a60-80cd-a375a748ef6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:20.728579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45620359-44d3-4a60-80cd-a375a748ef6e admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:20.729231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45620359-44d3-4a60-80cd-a375a748ef6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:20.731221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45620359-44d3-4a60-80cd-a375a748ef6e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:20.797678 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1068/2153] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:20 2022] POST /identity/v3/users => generated 283 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:20.798644 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45620359-44d3-4a60-80cd-a375a748ef6e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:20.800259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-45620359-44d3-4a60-80cd-a375a748ef6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:20.815217 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45620359-44d3-4a60-80cd-a375a748ef6e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:20.820791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72bc7d-88e6-4469-b81a-85a4039c4525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:20.821727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72bc7d-88e6-4469-b81a-85a4039c4525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:20.822087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72bc7d-88e6-4469-b81a-85a4039c4525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:20.822446 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72bc7d-88e6-4469-b81a-85a4039c4525 admin admin] PATH_INFO: `/v3/users/e59e46eb11b041c69c2bd241406c1dad` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:20.823665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72bc7d-88e6-4469-b81a-85a4039c4525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59e46eb11b041c69c2bd241406c1dad)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:20.825676 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72bc7d-88e6-4469-b81a-85a4039c4525 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:20.882147 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9affde803fd64a18a42341c89b820b7f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.945607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9affde803fd64a18a42341c89b820b7f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.958711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9affde803fd64a18a42341c89b820b7f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.965920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9affde803fd64a18a42341c89b820b7f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.966610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9affde803fd64a18a42341c89b820b7f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:20.975602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1086/2154] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:20 2022] DELETE /identity/v3/users/9affde803fd64a18a42341c89b820b7f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:20.975602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:20.977869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:21.012559 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5a72bc7d-88e6-4469-b81a-85a4039c4525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59e46eb11b041c69c2bd241406c1dad'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:21.049873 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2f781-b6d7-4b09-b6e7-388f2af24cae admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:21.054821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db81461-8ca0-4ff6-b9e4-4abb1302af1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:21.055467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db81461-8ca0-4ff6-b9e4-4abb1302af1a admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.055746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db81461-8ca0-4ff6-b9e4-4abb1302af1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.056045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db81461-8ca0-4ff6-b9e4-4abb1302af1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.056859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db81461-8ca0-4ff6-b9e4-4abb1302af1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:21.058581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db81461-8ca0-4ff6-b9e4-4abb1302af1a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:21.069193 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1087/2155] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:21.073408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db81461-8ca0-4ff6-b9e4-4abb1302af1a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:21.083147 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5a72bc7d-88e6-4469-b81a-85a4039c4525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59e46eb11b041c69c2bd241406c1dad'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:21.083890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db81461-8ca0-4ff6-b9e4-4abb1302af1a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:21.086206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:21.091056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8036858-555b-49e8-947e-a633a3539e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:21.091570 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8036858-555b-49e8-947e-a633a3539e87 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.091850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8036858-555b-49e8-947e-a633a3539e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.092207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8036858-555b-49e8-947e-a633a3539e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.092848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8036858-555b-49e8-947e-a633a3539e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:21.094569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8036858-555b-49e8-947e-a633a3539e87 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:21.096534 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5a72bc7d-88e6-4469-b81a-85a4039c4525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59e46eb11b041c69c2bd241406c1dad'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:21.097250 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5a72bc7d-88e6-4469-b81a-85a4039c4525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59e46eb11b041c69c2bd241406c1dad'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:21.097878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5a72bc7d-88e6-4469-b81a-85a4039c4525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59e46eb11b041c69c2bd241406c1dad'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:21.105249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1088/2156] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:21.111148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1069/2157] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:20 2022] DELETE /identity/v3/users/e59e46eb11b041c69c2bd241406c1dad => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:21.115001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8036858-555b-49e8-947e-a633a3539e87 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:21.116756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8c17a-5a12-4186-b2ef-6d350c1a004a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.116884 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a8036858-555b-49e8-947e-a633a3539e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:21.117042 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8c17a-5a12-4186-b2ef-6d350c1a004a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.117316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8c17a-5a12-4186-b2ef-6d350c1a004a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.129766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8036858-555b-49e8-947e-a633a3539e87 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:21.139242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-bbc8c17a-5a12-4186-b2ef-6d350c1a004a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:21.140528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bbc8c17a-5a12-4186-b2ef-6d350c1a004a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:21.151929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:21.205224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c54d7-7642-4c33-a806-aee2badabd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:21.205914 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c54d7-7642-4c33-a806-aee2badabd9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.206314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c54d7-7642-4c33-a806-aee2badabd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.206598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c54d7-7642-4c33-a806-aee2badabd9f admin admin] PATH_INFO: `/v3/projects/b21b4d6a06944eb1b61e3c180a5d0cdb/users/99803d82ea164c799eb77b5afed8d9e7/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.212199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1070/2158] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:21.222835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8c17a-5a12-4186-b2ef-6d350c1a004a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:21.223452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c54d7-7642-4c33-a806-aee2badabd9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21b4d6a06944eb1b61e3c180a5d0cdb, user_id=99803d82ea164c799eb77b5afed8d9e7, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:21.224824 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bbc8c17a-5a12-4186-b2ef-6d350c1a004a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:21.226294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c54d7-7642-4c33-a806-aee2badabd9f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:21.291265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8c17a-5a12-4186-b2ef-6d350c1a004a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:21.296268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1089/2159] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:21 2022] PUT /identity/v3/projects/b21b4d6a06944eb1b61e3c180a5d0cdb/users/99803d82ea164c799eb77b5afed8d9e7/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:21.317650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b37fb8-8927-4b6a-ac7b-f3a3d0373b3d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.317966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b37fb8-8927-4b6a-ac7b-f3a3d0373b3d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.318384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b37fb8-8927-4b6a-ac7b-f3a3d0373b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.351837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d6b37fb8-8927-4b6a-ac7b-f3a3d0373b3d None None] MFA Rules not processed for user `99803d82ea164c799eb77b5afed8d9e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:21.354747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d6b37fb8-8927-4b6a-ac7b-f3a3d0373b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:21.370645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177eba9-b8b0-4bee-80cd-c7c7ea2361b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:21.372427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177eba9-b8b0-4bee-80cd-c7c7ea2361b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.372711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177eba9-b8b0-4bee-80cd-c7c7ea2361b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.372981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177eba9-b8b0-4bee-80cd-c7c7ea2361b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.433667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1090/2160] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:21.434475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b37fb8-8927-4b6a-ac7b-f3a3d0373b3d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:21.435798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d6b37fb8-8927-4b6a-ac7b-f3a3d0373b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:21.450054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:21.455131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:21.470396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177eba9-b8b0-4bee-80cd-c7c7ea2361b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:21.471842 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177eba9-b8b0-4bee-80cd-c7c7ea2361b5 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:21.483163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1071/2161] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:39:21.484232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177eba9-b8b0-4bee-80cd-c7c7ea2361b5 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:21.495049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177eba9-b8b0-4bee-80cd-c7c7ea2361b5 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:21.505329 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b945af3-807f-4b7a-96e4-6d5b4fad062c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:21.507814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b945af3-807f-4b7a-96e4-6d5b4fad062c admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.509011 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b945af3-807f-4b7a-96e4-6d5b4fad062c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.509674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b945af3-807f-4b7a-96e4-6d5b4fad062c admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.511197 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b945af3-807f-4b7a-96e4-6d5b4fad062c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:21.514408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b945af3-807f-4b7a-96e4-6d5b4fad062c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:21.515510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b37fb8-8927-4b6a-ac7b-f3a3d0373b3d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:21.520870 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:21.521717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.521830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.522042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] PATH_INFO: `/v3/projects/95dbf683755b41a8b8b876a7d2f9e0a2` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.523196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=95dbf683755b41a8b8b876a7d2f9e0a2)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:21.525730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:21.644648 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1072/2162] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:21 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:21.648564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b945af3-807f-4b7a-96e4-6d5b4fad062c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:21.655671 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95dbf683755b41a8b8b876a7d2f9e0a2', 'request_id': 'req-807d10cc-77e3-45fb-8963-a3e50d271cef'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:21.662100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95dbf683755b41a8b8b876a7d2f9e0a2', 'request_id': 'req-807d10cc-77e3-45fb-8963-a3e50d271cef'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:21.721028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b945af3-807f-4b7a-96e4-6d5b4fad062c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:21.726538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b48db2-a331-4332-8e3e-80730f23045d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:21.727150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b48db2-a331-4332-8e3e-80730f23045d admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.727429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b48db2-a331-4332-8e3e-80730f23045d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.727700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b48db2-a331-4332-8e3e-80730f23045d admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.728367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b48db2-a331-4332-8e3e-80730f23045d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:21.730286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b48db2-a331-4332-8e3e-80730f23045d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:21.763781 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95dbf683755b41a8b8b876a7d2f9e0a2', 'request_id': 'req-807d10cc-77e3-45fb-8963-a3e50d271cef'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:21.779464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] The token cache is being invalidate because project 95dbf683755b41a8b8b876a7d2f9e0a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:21.779760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:21.780460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-807d10cc-77e3-45fb-8963-a3e50d271cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:21.784093 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1091/2163] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:21 2022] DELETE /identity/v3/projects/95dbf683755b41a8b8b876a7d2f9e0a2 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:21.787457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd1db5-abc0-45c7-8e5b-72de458af670 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.787561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd1db5-abc0-45c7-8e5b-72de458af670 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.787799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd1db5-abc0-45c7-8e5b-72de458af670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.810671 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b8cd1db5-abc0-45c7-8e5b-72de458af670 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:21.811889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b8cd1db5-abc0-45c7-8e5b-72de458af670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:21.819907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1073/2164] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:21 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:21.820341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1092/2165] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:21.821912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b48db2-a331-4332-8e3e-80730f23045d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:21.824044 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-78b48db2-a331-4332-8e3e-80730f23045d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:21.829587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153e86c-707f-40c6-a951-f5af4a7d6ea5 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.829878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153e86c-707f-40c6-a951-f5af4a7d6ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.830931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153e86c-707f-40c6-a951-f5af4a7d6ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.837380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b48db2-a331-4332-8e3e-80730f23045d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:21.848787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2273287-8248-48e5-9477-2c2badfccd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:21.849408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2273287-8248-48e5-9477-2c2badfccd84 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.849684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2273287-8248-48e5-9477-2c2badfccd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.849959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2273287-8248-48e5-9477-2c2badfccd84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.852109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2273287-8248-48e5-9477-2c2badfccd84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:21.852399 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2273287-8248-48e5-9477-2c2badfccd84 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:21.857163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9153e86c-707f-40c6-a951-f5af4a7d6ea5 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:21.861245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9153e86c-707f-40c6-a951-f5af4a7d6ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:21.864141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1074/2166] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:21.870020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2273287-8248-48e5-9477-2c2badfccd84 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:21.873634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1093/2167] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:21.877694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153e86c-707f-40c6-a951-f5af4a7d6ea5 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:21.881025 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2273287-8248-48e5-9477-2c2badfccd84 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:21.889435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153e86c-707f-40c6-a951-f5af4a7d6ea5 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:21.896374 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7845c2-eea7-4036-a727-c207ba90d449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:21.896754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5c4cb-12d6-416a-8956-7a17a5121ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:21.896845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7845c2-eea7-4036-a727-c207ba90d449 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.896845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7845c2-eea7-4036-a727-c207ba90d449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.896845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7845c2-eea7-4036-a727-c207ba90d449 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.897072 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5c4cb-12d6-416a-8956-7a17a5121ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.897072 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5c4cb-12d6-416a-8956-7a17a5121ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.897416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5c4cb-12d6-416a-8956-7a17a5121ac3 admin admin] PATH_INFO: `/v3/projects/aa24d3469d994c7ca3e8b671bc9a1b97/users/327eddec408d4d91b09d5fdab94b450f/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.897539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7845c2-eea7-4036-a727-c207ba90d449 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:21.897828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7845c2-eea7-4036-a727-c207ba90d449 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:21.900600 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7845c2-eea7-4036-a727-c207ba90d449 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:21.916928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1094/2168] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:21.921059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7845c2-eea7-4036-a727-c207ba90d449 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:21.930135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5c4cb-12d6-416a-8956-7a17a5121ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa24d3469d994c7ca3e8b671bc9a1b97, user_id=327eddec408d4d91b09d5fdab94b450f, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:21.931467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7845c2-eea7-4036-a727-c207ba90d449 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:21.933104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5c4cb-12d6-416a-8956-7a17a5121ac3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:21.942960 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4323c36-b837-4517-9dfd-6766e3d63589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:21.944058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323c36-b837-4517-9dfd-6766e3d63589 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:21.944058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323c36-b837-4517-9dfd-6766e3d63589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:21.944748 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323c36-b837-4517-9dfd-6766e3d63589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:21.944921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4323c36-b837-4517-9dfd-6766e3d63589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:21.946972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4323c36-b837-4517-9dfd-6766e3d63589 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:22.038619 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1075/2169] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:21 2022] PUT /identity/v3/projects/aa24d3469d994c7ca3e8b671bc9a1b97/users/327eddec408d4d91b09d5fdab94b450f/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:22.040374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5c4cb-12d6-416a-8956-7a17a5121ac3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:22.048266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b8a5c4cb-12d6-416a-8956-7a17a5121ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:22.061032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1095/2170] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:21 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:22.061032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4323c36-b837-4517-9dfd-6766e3d63589 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:22.121062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5c4cb-12d6-416a-8956-7a17a5121ac3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:22.146769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:22.153512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:22.158461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4323c36-b837-4517-9dfd-6766e3d63589 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:22.163293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942c7e6-08e0-48d3-b210-885849f1a898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:22.163898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942c7e6-08e0-48d3-b210-885849f1a898 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:22.164209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942c7e6-08e0-48d3-b210-885849f1a898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:22.165097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942c7e6-08e0-48d3-b210-885849f1a898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:22.165876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942c7e6-08e0-48d3-b210-885849f1a898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:22.167726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942c7e6-08e0-48d3-b210-885849f1a898 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:22.178351 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1096/2171] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:22.179314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942c7e6-08e0-48d3-b210-885849f1a898 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:22.186561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:22.187192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:22.187466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:22.187735 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] PATH_INFO: `/v3/projects/1ade396e1f1a43bf97c93dcb36f4d385` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:22.188972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ade396e1f1a43bf97c93dcb36f4d385)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:22.189179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942c7e6-08e0-48d3-b210-885849f1a898 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:22.190883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:22.194272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d3a06-91f1-43ac-95e8-2527f67abc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:22.194887 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d3a06-91f1-43ac-95e8-2527f67abc12 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:22.195169 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d3a06-91f1-43ac-95e8-2527f67abc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:22.197751 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d3a06-91f1-43ac-95e8-2527f67abc12 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:22.197751 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d3a06-91f1-43ac-95e8-2527f67abc12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:22.198575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d3a06-91f1-43ac-95e8-2527f67abc12 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:22.312396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1097/2172] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:22 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:22.313322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d3a06-91f1-43ac-95e8-2527f67abc12 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:22.323074 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d3a06-91f1-43ac-95e8-2527f67abc12 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:22.327670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acfe7a-34c4-4c96-9f22-791efcd5d80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:22.328325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acfe7a-34c4-4c96-9f22-791efcd5d80c admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:22.328607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acfe7a-34c4-4c96-9f22-791efcd5d80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:22.328881 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acfe7a-34c4-4c96-9f22-791efcd5d80c admin admin] PATH_INFO: `/v3/projects/aa24d3469d994c7ca3e8b671bc9a1b97/users/327eddec408d4d91b09d5fdab94b450f/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:22.330749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14acfe7a-34c4-4c96-9f22-791efcd5d80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa24d3469d994c7ca3e8b671bc9a1b97, user_id=327eddec408d4d91b09d5fdab94b450f, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:22.333468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14acfe7a-34c4-4c96-9f22-791efcd5d80c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:22.344894 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1098/2173] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:22 2022] PUT /identity/v3/projects/aa24d3469d994c7ca3e8b671bc9a1b97/users/327eddec408d4d91b09d5fdab94b450f/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:22.345684 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acfe7a-34c4-4c96-9f22-791efcd5d80c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:22.363135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ade396e1f1a43bf97c93dcb36f4d385', 'request_id': 'req-8153165d-8f62-495d-920e-781b1796b973'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:22.370397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ade396e1f1a43bf97c93dcb36f4d385', 'request_id': 'req-8153165d-8f62-495d-920e-781b1796b973'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:22.407652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acfe7a-34c4-4c96-9f22-791efcd5d80c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:22.454262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ade396e1f1a43bf97c93dcb36f4d385', 'request_id': 'req-8153165d-8f62-495d-920e-781b1796b973'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:22.464033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ab5e3-4edc-46af-80be-b93d127ab1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:22.464709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ab5e3-4edc-46af-80be-b93d127ab1bf admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:22.464998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ab5e3-4edc-46af-80be-b93d127ab1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:22.465380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ab5e3-4edc-46af-80be-b93d127ab1bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:22.466101 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ab5e3-4edc-46af-80be-b93d127ab1bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:22.466445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] The token cache is being invalidate because project 1ade396e1f1a43bf97c93dcb36f4d385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:22.466746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:22.467408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8153165d-8f62-495d-920e-781b1796b973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:22.467617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ab5e3-4edc-46af-80be-b93d127ab1bf admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:22.469012 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1076/2174] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:22 2022] DELETE /identity/v3/projects/1ade396e1f1a43bf97c93dcb36f4d385 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:22.470380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa8a12-00fb-492f-a1d4-780bf914997a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:22.470667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa8a12-00fb-492f-a1d4-780bf914997a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:22.470945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa8a12-00fb-492f-a1d4-780bf914997a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:22.483982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1099/2175] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:22.483982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ab5e3-4edc-46af-80be-b93d127ab1bf admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:22.485687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-245ab5e3-4edc-46af-80be-b93d127ab1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:22.499082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ab5e3-4edc-46af-80be-b93d127ab1bf admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:22.501750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c5fa8a12-00fb-492f-a1d4-780bf914997a None None] MFA Rules not processed for user `327eddec408d4d91b09d5fdab94b450f`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:22.502956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c5fa8a12-00fb-492f-a1d4-780bf914997a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:22.503707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d7b3d-1a23-4db1-94cf-cf7efb466050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:22.504350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d7b3d-1a23-4db1-94cf-cf7efb466050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:22.504629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d7b3d-1a23-4db1-94cf-cf7efb466050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:22.504945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d7b3d-1a23-4db1-94cf-cf7efb466050 admin admin] PATH_INFO: `/v3/users/a188179999d340838988242f9d61a874` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:22.506060 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d7b3d-1a23-4db1-94cf-cf7efb466050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a188179999d340838988242f9d61a874)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:22.507964 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d7b3d-1a23-4db1-94cf-cf7efb466050 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:22.575740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1077/2176] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:22.576688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa8a12-00fb-492f-a1d4-780bf914997a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:22.578698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c5fa8a12-00fb-492f-a1d4-780bf914997a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:22.595624 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa8a12-00fb-492f-a1d4-780bf914997a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:22.600659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47950d4f-51f3-4c2c-b447-b6ad44005584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:22.601876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47950d4f-51f3-4c2c-b447-b6ad44005584 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:22.601876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47950d4f-51f3-4c2c-b447-b6ad44005584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:22.601876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47950d4f-51f3-4c2c-b447-b6ad44005584 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:22.603334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47950d4f-51f3-4c2c-b447-b6ad44005584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:22.604977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47950d4f-51f3-4c2c-b447-b6ad44005584 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:22.615153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1078/2177] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:22.616137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47950d4f-51f3-4c2c-b447-b6ad44005584 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:22.617801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-47950d4f-51f3-4c2c-b447-b6ad44005584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:22.655653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:22.679232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47950d4f-51f3-4c2c-b447-b6ad44005584 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:22.683301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658c642-0c90-47a9-8223-51cf567ce202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:22.683908 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658c642-0c90-47a9-8223-51cf567ce202 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:22.684219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658c642-0c90-47a9-8223-51cf567ce202 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:22.684527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658c642-0c90-47a9-8223-51cf567ce202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:22.694809 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658c642-0c90-47a9-8223-51cf567ce202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:22.696996 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658c642-0c90-47a9-8223-51cf567ce202 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:22.707219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1079/2178] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:22.708077 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658c642-0c90-47a9-8223-51cf567ce202 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:22.708323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-824d7b3d-1a23-4db1-94cf-cf7efb466050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a188179999d340838988242f9d61a874'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:22.720370 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658c642-0c90-47a9-8223-51cf567ce202 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:22.725157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568c007-315f-44b5-be0c-88a42477cf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:22.725814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568c007-315f-44b5-be0c-88a42477cf56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:22.726207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568c007-315f-44b5-be0c-88a42477cf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:22.726489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568c007-315f-44b5-be0c-88a42477cf56 admin admin] PATH_INFO: `/v3/projects/764aad91598f4b13a6967b2d1fb7e59d/users/f240242958e34cbf81846b4e0cf4f437/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:22.743216 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568c007-315f-44b5-be0c-88a42477cf56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=764aad91598f4b13a6967b2d1fb7e59d, user_id=f240242958e34cbf81846b4e0cf4f437, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:22.746228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568c007-315f-44b5-be0c-88a42477cf56 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:22.757305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-824d7b3d-1a23-4db1-94cf-cf7efb466050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a188179999d340838988242f9d61a874'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:22.759431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:22.770800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-824d7b3d-1a23-4db1-94cf-cf7efb466050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a188179999d340838988242f9d61a874'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:22.777893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-824d7b3d-1a23-4db1-94cf-cf7efb466050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a188179999d340838988242f9d61a874'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:22.778585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-824d7b3d-1a23-4db1-94cf-cf7efb466050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a188179999d340838988242f9d61a874'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:22.784597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1100/2179] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:22 2022] DELETE /identity/v3/users/a188179999d340838988242f9d61a874 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:22.839677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1080/2180] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:22 2022] PUT /identity/v3/projects/764aad91598f4b13a6967b2d1fb7e59d/users/f240242958e34cbf81846b4e0cf4f437/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:22.844788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ca986-3542-46d4-b7ef-44ea13bc4e17 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:22.845323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ca986-3542-46d4-b7ef-44ea13bc4e17 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:22.845945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ca986-3542-46d4-b7ef-44ea13bc4e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:22.865331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-040ca986-3542-46d4-b7ef-44ea13bc4e17 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:22.872851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-040ca986-3542-46d4-b7ef-44ea13bc4e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:22.932491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1101/2181] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:22.945367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568c007-315f-44b5-be0c-88a42477cf56 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:22.948387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0568c007-315f-44b5-be0c-88a42477cf56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:23.010326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568c007-315f-44b5-be0c-88a42477cf56 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:23.014520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97060114-bb8c-4b33-8af8-7409e33b0f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:23.015120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97060114-bb8c-4b33-8af8-7409e33b0f02 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:23.015398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97060114-bb8c-4b33-8af8-7409e33b0f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:23.015785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97060114-bb8c-4b33-8af8-7409e33b0f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:23.025492 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97060114-bb8c-4b33-8af8-7409e33b0f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:23.026952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97060114-bb8c-4b33-8af8-7409e33b0f02 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:23.037266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1081/2182] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:23.251135 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ca986-3542-46d4-b7ef-44ea13bc4e17 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:23.254952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-040ca986-3542-46d4-b7ef-44ea13bc4e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:23.275370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ca986-3542-46d4-b7ef-44ea13bc4e17 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:23.281315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:23.281948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:23.282231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:23.282503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] PATH_INFO: `/v3/projects/db384fd64d40479ab4623345a938b101` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:23.283640 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db384fd64d40479ab4623345a938b101)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:23.285753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:23.426252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db384fd64d40479ab4623345a938b101', 'request_id': 'req-b98210ff-5c2d-4472-b725-63e8020cd557'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:23.432631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db384fd64d40479ab4623345a938b101', 'request_id': 'req-b98210ff-5c2d-4472-b725-63e8020cd557'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:23.552061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db384fd64d40479ab4623345a938b101', 'request_id': 'req-b98210ff-5c2d-4472-b725-63e8020cd557'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:23.564514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] The token cache is being invalidate because project db384fd64d40479ab4623345a938b101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:23.564808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:23.565585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b98210ff-5c2d-4472-b725-63e8020cd557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:23.572252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1102/2183] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:23 2022] DELETE /identity/v3/projects/db384fd64d40479ab4623345a938b101 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:23.572337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97060114-bb8c-4b33-8af8-7409e33b0f02 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:23.581319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97060114-bb8c-4b33-8af8-7409e33b0f02 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:23.651142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:23.651839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:23.652155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:23.652437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] PATH_INFO: `/v3/users/d988cbc7cf1a4a2a90bdc5c721edfa05` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:23.653740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d988cbc7cf1a4a2a90bdc5c721edfa05)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:23.655729 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:23.881382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd988cbc7cf1a4a2a90bdc5c721edfa05'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:23.985181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd988cbc7cf1a4a2a90bdc5c721edfa05'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:23.987534 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:24.001712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd988cbc7cf1a4a2a90bdc5c721edfa05'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:24.002555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd988cbc7cf1a4a2a90bdc5c721edfa05'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:24.003263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd988cbc7cf1a4a2a90bdc5c721edfa05'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:24.026958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1082/2184] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:23 2022] DELETE /identity/v3/users/d988cbc7cf1a4a2a90bdc5c721edfa05 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:24.354293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae240b0-776a-42e9-9f0e-906809834257 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:24.354293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae240b0-776a-42e9-9f0e-906809834257 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:24.354293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae240b0-776a-42e9-9f0e-906809834257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:24.418863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-dae240b0-776a-42e9-9f0e-906809834257 None None] MFA Rules not processed for user `f240242958e34cbf81846b4e0cf4f437`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:24.420470 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dae240b0-776a-42e9-9f0e-906809834257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:24.480579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:24.510107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:24.512132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:24.515706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1103/2185] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:24.520345 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae240b0-776a-42e9-9f0e-906809834257 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:24.522910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dae240b0-776a-42e9-9f0e-906809834257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:24.622592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae240b0-776a-42e9-9f0e-906809834257 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:24.624512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b501529-b3c4-42d0-9bd6-00c29ff8a821 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:24.627969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceeb46-8b0b-4bf2-9359-852061a0f645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:24.628611 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceeb46-8b0b-4bf2-9359-852061a0f645 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:24.628892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceeb46-8b0b-4bf2-9359-852061a0f645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:24.629168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceeb46-8b0b-4bf2-9359-852061a0f645 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:24.629709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:24.629818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dceeb46-8b0b-4bf2-9359-852061a0f645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:24.630497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:24.630786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:24.631060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] PATH_INFO: `/v3/projects/28914115b3724965b7b4dedc168bee3f` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:24.631837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dceeb46-8b0b-4bf2-9359-852061a0f645 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:24.632249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] RBAC: Authorizing `identity:delete_project(project_id=28914115b3724965b7b4dedc168bee3f)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:24.635673 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:24.707330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1104/2186] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:24 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:24.711931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceeb46-8b0b-4bf2-9359-852061a0f645 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:24.769713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28914115b3724965b7b4dedc168bee3f', 'request_id': 'req-ed842e43-0d82-447d-aa79-cb9ad36f382f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:24.773637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceeb46-8b0b-4bf2-9359-852061a0f645 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:24.777129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28914115b3724965b7b4dedc168bee3f', 'request_id': 'req-ed842e43-0d82-447d-aa79-cb9ad36f382f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:24.836189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3db951-f858-4c44-8039-b7971a47c29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:24.837637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3db951-f858-4c44-8039-b7971a47c29d admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:24.837932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3db951-f858-4c44-8039-b7971a47c29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:24.838180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3db951-f858-4c44-8039-b7971a47c29d admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:24.838808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3db951-f858-4c44-8039-b7971a47c29d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:24.840762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3db951-f858-4c44-8039-b7971a47c29d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:24.850211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28914115b3724965b7b4dedc168bee3f', 'request_id': 'req-ed842e43-0d82-447d-aa79-cb9ad36f382f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:24.862190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] The token cache is being invalidate because project 28914115b3724965b7b4dedc168bee3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:24.862481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:24.863150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed842e43-0d82-447d-aa79-cb9ad36f382f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:24.864802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1083/2187] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:24 2022] DELETE /identity/v3/projects/28914115b3724965b7b4dedc168bee3f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:24.871046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1efd60-03ad-4f59-b7cf-df0a82030986 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:24.871433 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1efd60-03ad-4f59-b7cf-df0a82030986 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:24.871592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1efd60-03ad-4f59-b7cf-df0a82030986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:24.891108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ca1efd60-03ad-4f59-b7cf-df0a82030986 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:24.892394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ca1efd60-03ad-4f59-b7cf-df0a82030986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:24.901266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1084/2188] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:24.909515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce4ee8-4335-4491-a577-accfe35aa71e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:24.912184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce4ee8-4335-4491-a577-accfe35aa71e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:24.912184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce4ee8-4335-4491-a577-accfe35aa71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:24.930635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f9ce4ee8-4335-4491-a577-accfe35aa71e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:24.931834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f9ce4ee8-4335-4491-a577-accfe35aa71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:24.940475 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1085/2189] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:24.944582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce4ee8-4335-4491-a577-accfe35aa71e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:24.953834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce4ee8-4335-4491-a577-accfe35aa71e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:24.958947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1b31f-f1aa-49fc-aef9-d1a34e4a634e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:24.959523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1b31f-f1aa-49fc-aef9-d1a34e4a634e admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:24.959859 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1b31f-f1aa-49fc-aef9-d1a34e4a634e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:24.960124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1b31f-f1aa-49fc-aef9-d1a34e4a634e admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:24.960775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1b31f-f1aa-49fc-aef9-d1a34e4a634e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:24.961081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1b31f-f1aa-49fc-aef9-d1a34e4a634e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:24.963259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1b31f-f1aa-49fc-aef9-d1a34e4a634e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:24.971033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1105/2190] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:24 2022] POST /identity/v3/users => generated 292 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:24.975812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3db951-f858-4c44-8039-b7971a47c29d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:24.978263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ad3db951-f858-4c44-8039-b7971a47c29d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:24.979746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1086/2191] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:39:24.982844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1b31f-f1aa-49fc-aef9-d1a34e4a634e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:24.998601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1b31f-f1aa-49fc-aef9-d1a34e4a634e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:25.003055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3db951-f858-4c44-8039-b7971a47c29d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:25.004878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee86b22-fab3-4844-8a76-f1648a8a9e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:25.005591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee86b22-fab3-4844-8a76-f1648a8a9e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:25.006061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee86b22-fab3-4844-8a76-f1648a8a9e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:25.006437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee86b22-fab3-4844-8a76-f1648a8a9e99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:25.007273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee86b22-fab3-4844-8a76-f1648a8a9e99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:25.010214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee86b22-fab3-4844-8a76-f1648a8a9e99 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:25.019311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d10a8c-d996-4f0d-a8f6-358ca3a588b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:25.024832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d10a8c-d996-4f0d-a8f6-358ca3a588b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:25.024832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d10a8c-d996-4f0d-a8f6-358ca3a588b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:25.024832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d10a8c-d996-4f0d-a8f6-358ca3a588b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:25.024832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d10a8c-d996-4f0d-a8f6-358ca3a588b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:25.024832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d10a8c-d996-4f0d-a8f6-358ca3a588b3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:25.036588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1106/2192] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:25.040749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d10a8c-d996-4f0d-a8f6-358ca3a588b3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:25.053498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d10a8c-d996-4f0d-a8f6-358ca3a588b3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:25.061342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf35675-f77c-4fca-910a-5a72bbf2a23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:25.062461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf35675-f77c-4fca-910a-5a72bbf2a23d admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:25.062993 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf35675-f77c-4fca-910a-5a72bbf2a23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:25.063404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf35675-f77c-4fca-910a-5a72bbf2a23d admin admin] PATH_INFO: `/v3/projects/1a5425bf731141088a5e23a4171918e4/users/50b3d596dc7345108154b9a4e7b6d4e8/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:25.099569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf35675-f77c-4fca-910a-5a72bbf2a23d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5425bf731141088a5e23a4171918e4, user_id=50b3d596dc7345108154b9a4e7b6d4e8, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:25.099569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf35675-f77c-4fca-910a-5a72bbf2a23d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:25.099741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1087/2193] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:24 2022] POST /identity/v3/projects => generated 399 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:25.103514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee86b22-fab3-4844-8a76-f1648a8a9e99 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:25.116886 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:25.166014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee86b22-fab3-4844-8a76-f1648a8a9e99 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:25.172738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc923d0-05c0-4c54-aaeb-0b028d89db5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:25.173351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc923d0-05c0-4c54-aaeb-0b028d89db5b admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:25.174158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc923d0-05c0-4c54-aaeb-0b028d89db5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:25.174158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc923d0-05c0-4c54-aaeb-0b028d89db5b admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:25.177100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc923d0-05c0-4c54-aaeb-0b028d89db5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:25.177261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc923d0-05c0-4c54-aaeb-0b028d89db5b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:25.177753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1107/2194] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:25 2022] PUT /identity/v3/projects/1a5425bf731141088a5e23a4171918e4/users/50b3d596dc7345108154b9a4e7b6d4e8/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:25.186444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf35675-f77c-4fca-910a-5a72bbf2a23d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:25.256536 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf35675-f77c-4fca-910a-5a72bbf2a23d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:25.263791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9cc42-5f49-49cf-b96d-4a990a9b6759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:25.263791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9cc42-5f49-49cf-b96d-4a990a9b6759 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:25.263791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9cc42-5f49-49cf-b96d-4a990a9b6759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:25.263791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9cc42-5f49-49cf-b96d-4a990a9b6759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:25.264275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9cc42-5f49-49cf-b96d-4a990a9b6759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:25.265865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9cc42-5f49-49cf-b96d-4a990a9b6759 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:25.276045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1108/2195] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:25.279891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9cc42-5f49-49cf-b96d-4a990a9b6759 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:25.290530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9cc42-5f49-49cf-b96d-4a990a9b6759 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:25.295563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b1af7-17ef-4de2-b49b-a1517cc61021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:25.296244 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b1af7-17ef-4de2-b49b-a1517cc61021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:25.296525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b1af7-17ef-4de2-b49b-a1517cc61021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:25.296807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b1af7-17ef-4de2-b49b-a1517cc61021 admin admin] PATH_INFO: `/v3/projects/1a5425bf731141088a5e23a4171918e4/users/50b3d596dc7345108154b9a4e7b6d4e8/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:25.298716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b1af7-17ef-4de2-b49b-a1517cc61021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5425bf731141088a5e23a4171918e4, user_id=50b3d596dc7345108154b9a4e7b6d4e8, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:25.301391 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b1af7-17ef-4de2-b49b-a1517cc61021 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:25.317441 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1109/2196] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:25 2022] PUT /identity/v3/projects/1a5425bf731141088a5e23a4171918e4/users/50b3d596dc7345108154b9a4e7b6d4e8/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:25.320875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1088/2197] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:25 2022] POST /identity/v3/users => generated 300 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:25.323710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b1af7-17ef-4de2-b49b-a1517cc61021 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:25.408785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b1af7-17ef-4de2-b49b-a1517cc61021 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:25.413704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af171a73-cdf5-4e4b-903c-a1c77cef59d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:25.414799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af171a73-cdf5-4e4b-903c-a1c77cef59d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:25.415173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af171a73-cdf5-4e4b-903c-a1c77cef59d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:25.415452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af171a73-cdf5-4e4b-903c-a1c77cef59d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:25.416214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af171a73-cdf5-4e4b-903c-a1c77cef59d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:25.417796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af171a73-cdf5-4e4b-903c-a1c77cef59d6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:25.428372 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1110/2198] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:25.433030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc923d0-05c0-4c54-aaeb-0b028d89db5b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:25.442996 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc923d0-05c0-4c54-aaeb-0b028d89db5b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:25.448638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616f59f-c085-4cd0-8590-a8bfffeef094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:25.449222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f59f-c085-4cd0-8590-a8bfffeef094 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:25.449498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f59f-c085-4cd0-8590-a8bfffeef094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:25.449768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f59f-c085-4cd0-8590-a8bfffeef094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:25.450409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616f59f-c085-4cd0-8590-a8bfffeef094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:25.451983 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616f59f-c085-4cd0-8590-a8bfffeef094 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:25.462631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1089/2199] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:25.466662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af171a73-cdf5-4e4b-903c-a1c77cef59d6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:25.476656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af171a73-cdf5-4e4b-903c-a1c77cef59d6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:25.481368 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cedaa-3cb1-4eb9-a2f3-48b6c071ca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:25.482101 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cedaa-3cb1-4eb9-a2f3-48b6c071ca24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:25.482475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cedaa-3cb1-4eb9-a2f3-48b6c071ca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:25.482817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cedaa-3cb1-4eb9-a2f3-48b6c071ca24 admin admin] PATH_INFO: `/v3/projects/f8b3b659d6b0455ebf5447e5563c3e15/users/509d6620f7bc42c798ea20d9e88917f1/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:25.498007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cedaa-3cb1-4eb9-a2f3-48b6c071ca24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b3b659d6b0455ebf5447e5563c3e15, user_id=509d6620f7bc42c798ea20d9e88917f1, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:25.500804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cedaa-3cb1-4eb9-a2f3-48b6c071ca24 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:25.609731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1111/2200] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:25 2022] PUT /identity/v3/projects/f8b3b659d6b0455ebf5447e5563c3e15/users/509d6620f7bc42c798ea20d9e88917f1/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:25.629392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0bed1-046a-4bae-ba68-569d9bab28cc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:25.629941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0bed1-046a-4bae-ba68-569d9bab28cc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:25.629941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0bed1-046a-4bae-ba68-569d9bab28cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:25.663782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-cfe0bed1-046a-4bae-ba68-569d9bab28cc None None] MFA Rules not processed for user `509d6620f7bc42c798ea20d9e88917f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:25.665404 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cfe0bed1-046a-4bae-ba68-569d9bab28cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:25.748605 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1090/2201] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:25 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:25.753098 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cedaa-3cb1-4eb9-a2f3-48b6c071ca24 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:25.816713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cedaa-3cb1-4eb9-a2f3-48b6c071ca24 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:25.822124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec115d-9d87-465a-83ca-39b206be59da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:25.822730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec115d-9d87-465a-83ca-39b206be59da admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:25.823013 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec115d-9d87-465a-83ca-39b206be59da admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:25.823289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec115d-9d87-465a-83ca-39b206be59da admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:25.823914 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec115d-9d87-465a-83ca-39b206be59da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:25.826005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec115d-9d87-465a-83ca-39b206be59da admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:25.973692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1112/2202] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:25 2022] POST /identity/v3/projects => generated 401 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:25.978124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0bed1-046a-4bae-ba68-569d9bab28cc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:25.989207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0bed1-046a-4bae-ba68-569d9bab28cc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:26.052738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52064879-4294-44bb-812d-6d1feb51ac7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:26.053407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52064879-4294-44bb-812d-6d1feb51ac7c admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:26.053710 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52064879-4294-44bb-812d-6d1feb51ac7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:26.054009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52064879-4294-44bb-812d-6d1feb51ac7c admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:26.054704 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52064879-4294-44bb-812d-6d1feb51ac7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:26.056867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52064879-4294-44bb-812d-6d1feb51ac7c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:26.125377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec115d-9d87-465a-83ca-39b206be59da admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:26.127135 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-beec115d-9d87-465a-83ca-39b206be59da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:26.140341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec115d-9d87-465a-83ca-39b206be59da admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:26.144877 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248b098-6270-4218-b91d-00051d8ac013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:26.145492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248b098-6270-4218-b91d-00051d8ac013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:26.145771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248b098-6270-4218-b91d-00051d8ac013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:26.146048 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248b098-6270-4218-b91d-00051d8ac013 admin admin] PATH_INFO: `/v3/users/99803d82ea164c799eb77b5afed8d9e7` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:26.147171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248b098-6270-4218-b91d-00051d8ac013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99803d82ea164c799eb77b5afed8d9e7)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:26.149913 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248b098-6270-4218-b91d-00051d8ac013 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:26.446995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:26.458263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1091/2203] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:25 2022] POST /identity/v3/users => generated 307 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:26.465929 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52064879-4294-44bb-812d-6d1feb51ac7c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:26.476349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52064879-4294-44bb-812d-6d1feb51ac7c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:26.482199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee9037-a56c-4f0e-b554-74ef70fb615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:26.482876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee9037-a56c-4f0e-b554-74ef70fb615f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:26.483224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee9037-a56c-4f0e-b554-74ef70fb615f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:26.483563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee9037-a56c-4f0e-b554-74ef70fb615f admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:26.484324 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee9037-a56c-4f0e-b554-74ef70fb615f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:26.486029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee9037-a56c-4f0e-b554-74ef70fb615f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:26.498421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1092/2204] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:26.502686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee9037-a56c-4f0e-b554-74ef70fb615f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:26.515744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee9037-a56c-4f0e-b554-74ef70fb615f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:26.520627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2c98e-99c4-4e3d-8000-f21ab88d7142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:26.521284 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2c98e-99c4-4e3d-8000-f21ab88d7142 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:26.521561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2c98e-99c4-4e3d-8000-f21ab88d7142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:26.522155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2c98e-99c4-4e3d-8000-f21ab88d7142 admin admin] PATH_INFO: `/v3/projects/0fe0b5ec1aad4bc5b5bea74b8fd23852/users/dccab8e799244d6ebf9076fe23a2c5e9/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:26.541381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2c98e-99c4-4e3d-8000-f21ab88d7142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe0b5ec1aad4bc5b5bea74b8fd23852, user_id=dccab8e799244d6ebf9076fe23a2c5e9, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:26.544739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2c98e-99c4-4e3d-8000-f21ab88d7142 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:26.570209 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0248b098-6270-4218-b91d-00051d8ac013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99803d82ea164c799eb77b5afed8d9e7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:26.593171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1093/2205] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:26 2022] PUT /identity/v3/projects/0fe0b5ec1aad4bc5b5bea74b8fd23852/users/dccab8e799244d6ebf9076fe23a2c5e9/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:26.597247 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2c98e-99c4-4e3d-8000-f21ab88d7142 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:26.649521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0248b098-6270-4218-b91d-00051d8ac013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99803d82ea164c799eb77b5afed8d9e7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:26.663948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0248b098-6270-4218-b91d-00051d8ac013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99803d82ea164c799eb77b5afed8d9e7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:26.672075 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0248b098-6270-4218-b91d-00051d8ac013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99803d82ea164c799eb77b5afed8d9e7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:26.672863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0248b098-6270-4218-b91d-00051d8ac013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99803d82ea164c799eb77b5afed8d9e7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:26.678235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2c98e-99c4-4e3d-8000-f21ab88d7142 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:26.680423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1113/2206] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:26 2022] DELETE /identity/v3/users/99803d82ea164c799eb77b5afed8d9e7 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:26.688621 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb082255-e354-4823-b45f-024c1a4b3e47 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:26.688982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb082255-e354-4823-b45f-024c1a4b3e47 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:26.689334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb082255-e354-4823-b45f-024c1a4b3e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:26.745114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cb082255-e354-4823-b45f-024c1a4b3e47 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:26.747857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cb082255-e354-4823-b45f-024c1a4b3e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:26.789347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a6e2f-6573-4c75-bba7-6c600ec93643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:26.790593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a6e2f-6573-4c75-bba7-6c600ec93643 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:26.791046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a6e2f-6573-4c75-bba7-6c600ec93643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:26.791577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a6e2f-6573-4c75-bba7-6c600ec93643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:26.792870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a6e2f-6573-4c75-bba7-6c600ec93643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:26.795156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a6e2f-6573-4c75-bba7-6c600ec93643 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:26.807876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1094/2207] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:26.812203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a6e2f-6573-4c75-bba7-6c600ec93643 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:26.814709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-365a6e2f-6573-4c75-bba7-6c600ec93643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:26.829954 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a6e2f-6573-4c75-bba7-6c600ec93643 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:26.835256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eaa7c-fbb9-477a-a050-25a0b3716c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:26.836204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eaa7c-fbb9-477a-a050-25a0b3716c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:26.840426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eaa7c-fbb9-477a-a050-25a0b3716c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:26.840426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eaa7c-fbb9-477a-a050-25a0b3716c15 admin admin] PATH_INFO: `/v3/projects/0fe0b5ec1aad4bc5b5bea74b8fd23852/users/dccab8e799244d6ebf9076fe23a2c5e9/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:26.840426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963eaa7c-fbb9-477a-a050-25a0b3716c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe0b5ec1aad4bc5b5bea74b8fd23852, user_id=dccab8e799244d6ebf9076fe23a2c5e9, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:26.842222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963eaa7c-fbb9-477a-a050-25a0b3716c15 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:26.854523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1095/2208] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:26 2022] PUT /identity/v3/projects/0fe0b5ec1aad4bc5b5bea74b8fd23852/users/dccab8e799244d6ebf9076fe23a2c5e9/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:26.860193 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1114/2209] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:26.862221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eaa7c-fbb9-477a-a050-25a0b3716c15 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:26.864326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-963eaa7c-fbb9-477a-a050-25a0b3716c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:26.866172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1847-c328-4118-ac26-c8fc7564567f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:26.866569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1847-c328-4118-ac26-c8fc7564567f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:26.866914 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1847-c328-4118-ac26-c8fc7564567f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:26.896583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-671b1847-c328-4118-ac26-c8fc7564567f None None] MFA Rules not processed for user `dccab8e799244d6ebf9076fe23a2c5e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:26.897922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-671b1847-c328-4118-ac26-c8fc7564567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:26.929987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eaa7c-fbb9-477a-a050-25a0b3716c15 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:26.935072 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273313c-3e92-4293-8b3c-fb461e54d4fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:26.935700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273313c-3e92-4293-8b3c-fb461e54d4fb service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:26.936549 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273313c-3e92-4293-8b3c-fb461e54d4fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:26.936549 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273313c-3e92-4293-8b3c-fb461e54d4fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:26.994991 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1115/2210] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:26 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:27.006725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1847-c328-4118-ac26-c8fc7564567f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:27.009290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-671b1847-c328-4118-ac26-c8fc7564567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:27.010437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273313c-3e92-4293-8b3c-fb461e54d4fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:27.011954 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273313c-3e92-4293-8b3c-fb461e54d4fb service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:27.040933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1096/2211] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:27.088310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1847-c328-4118-ac26-c8fc7564567f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:27.096228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ab19d-2f01-4380-a30b-17143df5a17b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:27.096228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ab19d-2f01-4380-a30b-17143df5a17b service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:27.096228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ab19d-2f01-4380-a30b-17143df5a17b service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:27.096228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ab19d-2f01-4380-a30b-17143df5a17b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:27.109280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ab19d-2f01-4380-a30b-17143df5a17b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:27.111181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ab19d-2f01-4380-a30b-17143df5a17b service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:27.123924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1116/2212] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:27 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:39:27.323883 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:27.390635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273313c-3e92-4293-8b3c-fb461e54d4fb service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:27.400437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273313c-3e92-4293-8b3c-fb461e54d4fb service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:27.404102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c249369-ccd2-43b0-bd5a-616cda5ad390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:27.404738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c249369-ccd2-43b0-bd5a-616cda5ad390 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:27.405013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c249369-ccd2-43b0-bd5a-616cda5ad390 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:27.405435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c249369-ccd2-43b0-bd5a-616cda5ad390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:27.407193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0c249369-ccd2-43b0-bd5a-616cda5ad390 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:27.481873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c249369-ccd2-43b0-bd5a-616cda5ad390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:27.483234 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c249369-ccd2-43b0-bd5a-616cda5ad390 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:27.495740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1097/2213] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:27 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:27.543462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:27.550294 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ab19d-2f01-4380-a30b-17143df5a17b service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:27.553922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-447ab19d-2f01-4380-a30b-17143df5a17b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:27.570816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fab80-7a1f-4dc8-80b3-3ea1ef728f11 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:27.571175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fab80-7a1f-4dc8-80b3-3ea1ef728f11 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:27.571516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fab80-7a1f-4dc8-80b3-3ea1ef728f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:27.587161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ab19d-2f01-4380-a30b-17143df5a17b service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:27.601499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:27.603727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:27.604240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:27.604944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] PATH_INFO: `/v3/projects/b21b4d6a06944eb1b61e3c180a5d0cdb` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:27.607892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e09fab80-7a1f-4dc8-80b3-3ea1ef728f11 None None] MFA Rules not processed for user `50b3d596dc7345108154b9a4e7b6d4e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:27.608649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e09fab80-7a1f-4dc8-80b3-3ea1ef728f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:27.622338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21b4d6a06944eb1b61e3c180a5d0cdb)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:27.628364 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:27.713913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1098/2214] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:27.786803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21b4d6a06944eb1b61e3c180a5d0cdb', 'request_id': 'req-3bf3cd21-832f-4ba6-981b-138b2709305c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:27.793917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21b4d6a06944eb1b61e3c180a5d0cdb', 'request_id': 'req-3bf3cd21-832f-4ba6-981b-138b2709305c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:27.911763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21b4d6a06944eb1b61e3c180a5d0cdb', 'request_id': 'req-3bf3cd21-832f-4ba6-981b-138b2709305c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:27.925265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] The token cache is being invalidate because project b21b4d6a06944eb1b61e3c180a5d0cdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:27.925566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:27.926222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:27.927769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fab80-7a1f-4dc8-80b3-3ea1ef728f11 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:27.931746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1117/2215] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:27 2022] DELETE /identity/v3/projects/b21b4d6a06944eb1b61e3c180a5d0cdb => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:27.931874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e09fab80-7a1f-4dc8-80b3-3ea1ef728f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:27.932868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:27.934404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:27.950089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fab80-7a1f-4dc8-80b3-3ea1ef728f11 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:27.998780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02ea51-9dfa-417f-80ee-cc210d2c9c32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:27.999227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3cd21-832f-4ba6-981b-138b2709305c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:27.999341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02ea51-9dfa-417f-80ee-cc210d2c9c32 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:27.999577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02ea51-9dfa-417f-80ee-cc210d2c9c32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:27.999849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02ea51-9dfa-417f-80ee-cc210d2c9c32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:28.001587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ca02ea51-9dfa-417f-80ee-cc210d2c9c32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:28.003876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d9f97-62a3-4d67-bdfe-3f1a50b954c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:28.004540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d9f97-62a3-4d67-bdfe-3f1a50b954c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:28.004819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d9f97-62a3-4d67-bdfe-3f1a50b954c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:28.010850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d9f97-62a3-4d67-bdfe-3f1a50b954c0 admin admin] PATH_INFO: `/v3/users/327eddec408d4d91b09d5fdab94b450f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:28.010850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d9f97-62a3-4d67-bdfe-3f1a50b954c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327eddec408d4d91b09d5fdab94b450f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:28.010850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d9f97-62a3-4d67-bdfe-3f1a50b954c0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:28.065299 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02ea51-9dfa-417f-80ee-cc210d2c9c32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:28.066816 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02ea51-9dfa-417f-80ee-cc210d2c9c32 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:28.076042 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1099/2216] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:28.100328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:28.172119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-454d9f97-62a3-4d67-bdfe-3f1a50b954c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327eddec408d4d91b09d5fdab94b450f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:28.186220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:28.257671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02ea51-9dfa-417f-80ee-cc210d2c9c32 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:28.260683 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ca02ea51-9dfa-417f-80ee-cc210d2c9c32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:28.275139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02ea51-9dfa-417f-80ee-cc210d2c9c32 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:28.281453 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:28.284180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-454d9f97-62a3-4d67-bdfe-3f1a50b954c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327eddec408d4d91b09d5fdab94b450f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:28.287371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:28.291332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:28.292937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] PATH_INFO: `/v3/users/90bad0adb01a4d34876826ac6fa18551` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:28.295612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] RBAC: Authorizing `identity:delete_user(user_id=90bad0adb01a4d34876826ac6fa18551)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:28.300634 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:28.304543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-454d9f97-62a3-4d67-bdfe-3f1a50b954c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327eddec408d4d91b09d5fdab94b450f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:28.314449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-454d9f97-62a3-4d67-bdfe-3f1a50b954c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327eddec408d4d91b09d5fdab94b450f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:28.315190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-454d9f97-62a3-4d67-bdfe-3f1a50b954c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327eddec408d4d91b09d5fdab94b450f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:28.321716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1118/2217] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:27 2022] DELETE /identity/v3/users/327eddec408d4d91b09d5fdab94b450f => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:28.555787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90bad0adb01a4d34876826ac6fa18551'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:28.660304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90bad0adb01a4d34876826ac6fa18551'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:28.678260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90bad0adb01a4d34876826ac6fa18551'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:28.678922 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90bad0adb01a4d34876826ac6fa18551'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:28.679740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90bad0adb01a4d34876826ac6fa18551'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:28.696209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1100/2218] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:28 2022] DELETE /identity/v3/users/90bad0adb01a4d34876826ac6fa18551 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:28.701473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbcd87-d963-4b41-af22-6c964ef3e1af None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:28.703278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbcd87-d963-4b41-af22-6c964ef3e1af None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:28.703592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbcd87-d963-4b41-af22-6c964ef3e1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:28.725729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f3dbcd87-d963-4b41-af22-6c964ef3e1af None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:28.727427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f3dbcd87-d963-4b41-af22-6c964ef3e1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:28.801541 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1119/2219] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:28.816140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:28.818029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:28.881743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38147a5d-2b8d-44e1-b879-d394e609c78e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:28.887221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d00fb2-2566-46ac-8381-71a79becc1ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:28.887984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d00fb2-2566-46ac-8381-71a79becc1ed service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:28.888371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d00fb2-2566-46ac-8381-71a79becc1ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:28.888856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d00fb2-2566-46ac-8381-71a79becc1ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:28.900742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d00fb2-2566-46ac-8381-71a79becc1ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:28.902441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d00fb2-2566-46ac-8381-71a79becc1ed service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:28.907934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:28.928235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1101/2220] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:29.017007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbcd87-d963-4b41-af22-6c964ef3e1af None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:29.018521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f3dbcd87-d963-4b41-af22-6c964ef3e1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:29.033873 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbcd87-d963-4b41-af22-6c964ef3e1af None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:29.041967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:29.042995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:29.043281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:29.043559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] PATH_INFO: `/v3/projects/aa24d3469d994c7ca3e8b671bc9a1b97` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:29.045542 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa24d3469d994c7ca3e8b671bc9a1b97)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:29.048493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:29.171997 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa24d3469d994c7ca3e8b671bc9a1b97', 'request_id': 'req-a99f31ff-bf85-43ac-9c46-b6dde457394a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:29.179149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa24d3469d994c7ca3e8b671bc9a1b97', 'request_id': 'req-a99f31ff-bf85-43ac-9c46-b6dde457394a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:29.237248 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa24d3469d994c7ca3e8b671bc9a1b97', 'request_id': 'req-a99f31ff-bf85-43ac-9c46-b6dde457394a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:29.263016 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] The token cache is being invalidate because project aa24d3469d994c7ca3e8b671bc9a1b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:29.263016 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:29.263016 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:29.268882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1120/2221] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:29 2022] DELETE /identity/v3/projects/aa24d3469d994c7ca3e8b671bc9a1b97 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:29.270669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965e2e0-1851-433e-baf8-57bb8ab1d788 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:29.271001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965e2e0-1851-433e-baf8-57bb8ab1d788 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:29.271285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965e2e0-1851-433e-baf8-57bb8ab1d788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:29.441844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2965e2e0-1851-433e-baf8-57bb8ab1d788 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:29.442260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2965e2e0-1851-433e-baf8-57bb8ab1d788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:29.446802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:29.452782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:29.456087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:29.486799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:29.584070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1102/2222] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:29.584280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f31ff-bf85-43ac-9c46-b6dde457394a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:29.587088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1de2d-6bad-40f4-8619-072ed39a9060 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:29.587442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1de2d-6bad-40f4-8619-072ed39a9060 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:29.587825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1de2d-6bad-40f4-8619-072ed39a9060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:29.593799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:29.594532 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:29.594831 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:29.595247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] PATH_INFO: `/v3/projects/e36146acdd4946399eb3bce115af6b1c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:29.599166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e36146acdd4946399eb3bce115af6b1c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:29.602804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:29.618315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8ed1de2d-6bad-40f4-8619-072ed39a9060 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:29.621657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8ed1de2d-6bad-40f4-8619-072ed39a9060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:29.631367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1103/2223] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:29.638791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1de2d-6bad-40f4-8619-072ed39a9060 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:29.650811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1de2d-6bad-40f4-8619-072ed39a9060 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:29.656858 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb256b-43e3-4c23-83f2-705e2fb05543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:29.657529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb256b-43e3-4c23-83f2-705e2fb05543 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:29.657819 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb256b-43e3-4c23-83f2-705e2fb05543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:29.658075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb256b-43e3-4c23-83f2-705e2fb05543 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:29.658782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb256b-43e3-4c23-83f2-705e2fb05543 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:29.659084 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb256b-43e3-4c23-83f2-705e2fb05543 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:29.660884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb256b-43e3-4c23-83f2-705e2fb05543 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:29.674793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1104/2224] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:29.679161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb256b-43e3-4c23-83f2-705e2fb05543 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:29.694119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb256b-43e3-4c23-83f2-705e2fb05543 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:29.713823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387d82c-c35a-4624-8f84-685b9b906217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:29.715222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387d82c-c35a-4624-8f84-685b9b906217 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:29.715644 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387d82c-c35a-4624-8f84-685b9b906217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:29.715987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387d82c-c35a-4624-8f84-685b9b906217 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:29.716712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387d82c-c35a-4624-8f84-685b9b906217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:29.718617 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387d82c-c35a-4624-8f84-685b9b906217 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:29.735247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e36146acdd4946399eb3bce115af6b1c', 'request_id': 'req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:29.742017 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e36146acdd4946399eb3bce115af6b1c', 'request_id': 'req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:29.796950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1105/2225] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:29 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:29.800978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387d82c-c35a-4624-8f84-685b9b906217 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:29.821221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e36146acdd4946399eb3bce115af6b1c', 'request_id': 'req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:29.833055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] The token cache is being invalidate because project e36146acdd4946399eb3bce115af6b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:29.833354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:29.834152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8650e7bd-7fe9-49fb-9e27-7a57b78199a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:29.838030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1121/2226] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:29 2022] DELETE /identity/v3/projects/e36146acdd4946399eb3bce115af6b1c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:29.843227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06303055-82d4-411e-80c2-04ea26dfd1ad None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:29.843620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06303055-82d4-411e-80c2-04ea26dfd1ad None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:29.845431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06303055-82d4-411e-80c2-04ea26dfd1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:29.869730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-06303055-82d4-411e-80c2-04ea26dfd1ad None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:29.871236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-06303055-82d4-411e-80c2-04ea26dfd1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:29.879799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387d82c-c35a-4624-8f84-685b9b906217 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:29.880507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1122/2227] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:29.896798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751cba4-1db8-47b6-8bad-594210764547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:29.897427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751cba4-1db8-47b6-8bad-594210764547 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:29.897709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751cba4-1db8-47b6-8bad-594210764547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:29.897981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751cba4-1db8-47b6-8bad-594210764547 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:29.899820 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751cba4-1db8-47b6-8bad-594210764547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:29.902416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751cba4-1db8-47b6-8bad-594210764547 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:29.903622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a57b2-7255-4b20-bf9e-7140315b14ef None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:29.904001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a57b2-7255-4b20-bf9e-7140315b14ef None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:29.904815 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a57b2-7255-4b20-bf9e-7140315b14ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:29.926750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-863a57b2-7255-4b20-bf9e-7140315b14ef None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:29.928000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-863a57b2-7255-4b20-bf9e-7140315b14ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:29.952311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1123/2228] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:29.952311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a57b2-7255-4b20-bf9e-7140315b14ef None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:29.956240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-863a57b2-7255-4b20-bf9e-7140315b14ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:30.059853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a57b2-7255-4b20-bf9e-7140315b14ef None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.062592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1106/2229] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:29 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:30.063598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2273a74-7f3e-4a10-98d1-2a6c5193d068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.064286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2273a74-7f3e-4a10-98d1-2a6c5193d068 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.064567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2273a74-7f3e-4a10-98d1-2a6c5193d068 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.064994 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2273a74-7f3e-4a10-98d1-2a6c5193d068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.066466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751cba4-1db8-47b6-8bad-594210764547 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.072837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d2273a74-7f3e-4a10-98d1-2a6c5193d068 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:30.077176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751cba4-1db8-47b6-8bad-594210764547 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.081872 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d1e3-58a7-416c-b95e-4704cfeffc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.083068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d1e3-58a7-416c-b95e-4704cfeffc05 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.083068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d1e3-58a7-416c-b95e-4704cfeffc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.083068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d1e3-58a7-416c-b95e-4704cfeffc05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.083690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261d1e3-58a7-416c-b95e-4704cfeffc05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:30.083993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261d1e3-58a7-416c-b95e-4704cfeffc05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.085595 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261d1e3-58a7-416c-b95e-4704cfeffc05 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.098202 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1107/2230] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:30.099441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d1e3-58a7-416c-b95e-4704cfeffc05 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.100719 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0261d1e3-58a7-416c-b95e-4704cfeffc05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:30.114027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d1e3-58a7-416c-b95e-4704cfeffc05 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.119585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699858a-b307-4249-868a-13e40ed89218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.120205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699858a-b307-4249-868a-13e40ed89218 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.120482 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699858a-b307-4249-868a-13e40ed89218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.120749 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699858a-b307-4249-868a-13e40ed89218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.121410 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699858a-b307-4249-868a-13e40ed89218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.123280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699858a-b307-4249-868a-13e40ed89218 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.134410 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1108/2231] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:30.135242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699858a-b307-4249-868a-13e40ed89218 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.144699 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2273a74-7f3e-4a10-98d1-2a6c5193d068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.145583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699858a-b307-4249-868a-13e40ed89218 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.147082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2273a74-7f3e-4a10-98d1-2a6c5193d068 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.151427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8c47f-c791-4357-bdeb-45911c6b63ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.152171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8c47f-c791-4357-bdeb-45911c6b63ff admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.152451 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8c47f-c791-4357-bdeb-45911c6b63ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.152723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8c47f-c791-4357-bdeb-45911c6b63ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.153514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8c47f-c791-4357-bdeb-45911c6b63ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.155428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8c47f-c791-4357-bdeb-45911c6b63ff admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.158375 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1124/2232] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:30.159251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2273a74-7f3e-4a10-98d1-2a6c5193d068 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.170469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2273a74-7f3e-4a10-98d1-2a6c5193d068 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.176369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ba5b9-5405-4908-8a90-f73ca877ad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.181054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ba5b9-5405-4908-8a90-f73ca877ad86 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.181054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ba5b9-5405-4908-8a90-f73ca877ad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.181054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ba5b9-5405-4908-8a90-f73ca877ad86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.181054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ba5b9-5405-4908-8a90-f73ca877ad86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.181054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ba5b9-5405-4908-8a90-f73ca877ad86 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.191078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1125/2233] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:30.195395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ba5b9-5405-4908-8a90-f73ca877ad86 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.205994 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ba5b9-5405-4908-8a90-f73ca877ad86 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.210747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d37034-866f-4139-8221-0043a3f8b9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.211927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d37034-866f-4139-8221-0043a3f8b9c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.212242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d37034-866f-4139-8221-0043a3f8b9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.212513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d37034-866f-4139-8221-0043a3f8b9c9 admin admin] PATH_INFO: `/v3/projects/3ec4880f316c4710a5aab05ac78a1836/users/1084064a53aa4dba9b14b23a3f485298/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.228998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d37034-866f-4139-8221-0043a3f8b9c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec4880f316c4710a5aab05ac78a1836, user_id=1084064a53aa4dba9b14b23a3f485298, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.231862 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d37034-866f-4139-8221-0043a3f8b9c9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.233903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1109/2234] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:30 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:30.237856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8c47f-c791-4357-bdeb-45911c6b63ff admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.253379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:30.289928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1126/2235] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:30 2022] PUT /identity/v3/projects/3ec4880f316c4710a5aab05ac78a1836/users/1084064a53aa4dba9b14b23a3f485298/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:30.301352 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01436a11-6219-4bc9-8b80-1cd366365ff3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.301651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01436a11-6219-4bc9-8b80-1cd366365ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.301909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01436a11-6219-4bc9-8b80-1cd366365ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.323558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:30.330992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8c47f-c791-4357-bdeb-45911c6b63ff admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.347200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-01436a11-6219-4bc9-8b80-1cd366365ff3 None None] MFA Rules not processed for user `1084064a53aa4dba9b14b23a3f485298`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:30.348615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-01436a11-6219-4bc9-8b80-1cd366365ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:30.421360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047d0e7-7aaa-40ef-b90a-0e1e6e1adf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.421972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047d0e7-7aaa-40ef-b90a-0e1e6e1adf88 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.422253 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047d0e7-7aaa-40ef-b90a-0e1e6e1adf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.422665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047d0e7-7aaa-40ef-b90a-0e1e6e1adf88 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.423305 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047d0e7-7aaa-40ef-b90a-0e1e6e1adf88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.425362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047d0e7-7aaa-40ef-b90a-0e1e6e1adf88 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.441264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1127/2236] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:30.446494 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01436a11-6219-4bc9-8b80-1cd366365ff3 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.459211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01436a11-6219-4bc9-8b80-1cd366365ff3 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.465340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de10f7-4c14-4847-92af-a249f1da824b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.466119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de10f7-4c14-4847-92af-a249f1da824b admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.466507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de10f7-4c14-4847-92af-a249f1da824b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.466906 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de10f7-4c14-4847-92af-a249f1da824b admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.467882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de10f7-4c14-4847-92af-a249f1da824b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.470641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de10f7-4c14-4847-92af-a249f1da824b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.541040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1110/2237] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:30 2022] POST /identity/v3/users => generated 288 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:30.545148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047d0e7-7aaa-40ef-b90a-0e1e6e1adf88 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.555636 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047d0e7-7aaa-40ef-b90a-0e1e6e1adf88 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.560442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de5e2a-3e8b-4e73-9edc-61f72239037b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.561035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de5e2a-3e8b-4e73-9edc-61f72239037b admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.561380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de5e2a-3e8b-4e73-9edc-61f72239037b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.561606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de5e2a-3e8b-4e73-9edc-61f72239037b admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.562387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de5e2a-3e8b-4e73-9edc-61f72239037b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.563990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de5e2a-3e8b-4e73-9edc-61f72239037b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.582771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1111/2238] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:30.583861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de5e2a-3e8b-4e73-9edc-61f72239037b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.592346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1128/2239] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:30 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:30.595517 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de10f7-4c14-4847-92af-a249f1da824b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.599015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de5e2a-3e8b-4e73-9edc-61f72239037b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.599805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c4de10f7-4c14-4847-92af-a249f1da824b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:30.674223 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7677117-fe50-414b-b3d7-efb019cabcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.674841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7677117-fe50-414b-b3d7-efb019cabcef admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.675123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7677117-fe50-414b-b3d7-efb019cabcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.675293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de10f7-4c14-4847-92af-a249f1da824b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.675395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7677117-fe50-414b-b3d7-efb019cabcef admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.676195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7677117-fe50-414b-b3d7-efb019cabcef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.677783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7677117-fe50-414b-b3d7-efb019cabcef admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.680860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902084a3-43e3-4423-8c02-bed74075e9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.681555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902084a3-43e3-4423-8c02-bed74075e9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.681820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902084a3-43e3-4423-8c02-bed74075e9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.682092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902084a3-43e3-4423-8c02-bed74075e9c8 admin admin] PATH_INFO: `/v3/users/7134f955e61b45fa8b8377696d3d1fd3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.683305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902084a3-43e3-4423-8c02-bed74075e9c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7134f955e61b45fa8b8377696d3d1fd3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.685246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902084a3-43e3-4423-8c02-bed74075e9c8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.693106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1112/2240] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:30.693106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7677117-fe50-414b-b3d7-efb019cabcef admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.704041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7677117-fe50-414b-b3d7-efb019cabcef admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.710571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450803f-8a55-4329-bede-167db4ab8d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.728463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450803f-8a55-4329-bede-167db4ab8d42 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.728463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450803f-8a55-4329-bede-167db4ab8d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.728463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450803f-8a55-4329-bede-167db4ab8d42 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.728463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450803f-8a55-4329-bede-167db4ab8d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.728463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450803f-8a55-4329-bede-167db4ab8d42 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.793952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1113/2241] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:30 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:30.794892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450803f-8a55-4329-bede-167db4ab8d42 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.807431 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450803f-8a55-4329-bede-167db4ab8d42 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:30.812590 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f430f9b-6777-4d7d-a2d0-5e8855541e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:30.813248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f430f9b-6777-4d7d-a2d0-5e8855541e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.813535 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f430f9b-6777-4d7d-a2d0-5e8855541e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.813814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f430f9b-6777-4d7d-a2d0-5e8855541e8e admin admin] PATH_INFO: `/v3/projects/d983573bf6424422a28e763405a98c7e/users/9c90c556d4c04951819794cf1cc11493/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:30.831328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:30.831328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f430f9b-6777-4d7d-a2d0-5e8855541e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d983573bf6424422a28e763405a98c7e, user_id=9c90c556d4c04951819794cf1cc11493, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:30.833681 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f430f9b-6777-4d7d-a2d0-5e8855541e8e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:30.880787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-902084a3-43e3-4423-8c02-bed74075e9c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7134f955e61b45fa8b8377696d3d1fd3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:30.911743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1114/2242] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:30 2022] PUT /identity/v3/projects/d983573bf6424422a28e763405a98c7e/users/9c90c556d4c04951819794cf1cc11493/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:30.913277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f430f9b-6777-4d7d-a2d0-5e8855541e8e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:30.924272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-902084a3-43e3-4423-8c02-bed74075e9c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7134f955e61b45fa8b8377696d3d1fd3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:30.946500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-902084a3-43e3-4423-8c02-bed74075e9c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7134f955e61b45fa8b8377696d3d1fd3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:30.955705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-902084a3-43e3-4423-8c02-bed74075e9c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7134f955e61b45fa8b8377696d3d1fd3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:30.956746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-902084a3-43e3-4423-8c02-bed74075e9c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7134f955e61b45fa8b8377696d3d1fd3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:30.966333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1129/2243] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:30 2022] DELETE /identity/v3/users/7134f955e61b45fa8b8377696d3d1fd3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:30.966333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584024e-7590-40ee-9652-5c0617675585 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:30.966333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584024e-7590-40ee-9652-5c0617675585 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:30.966333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584024e-7590-40ee-9652-5c0617675585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.004320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6584024e-7590-40ee-9652-5c0617675585 None None] MFA Rules not processed for user `9c90c556d4c04951819794cf1cc11493`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:31.004320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6584024e-7590-40ee-9652-5c0617675585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:31.018452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f430f9b-6777-4d7d-a2d0-5e8855541e8e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:31.042571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:31.100433 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1130/2244] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:31.102337 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd79c09-014a-4068-b020-d522c217a359 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:31.102622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd79c09-014a-4068-b020-d522c217a359 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:31.102897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd79c09-014a-4068-b020-d522c217a359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.106263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54629140-867c-4a75-8eb8-9f5b1195637f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:31.110671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54629140-867c-4a75-8eb8-9f5b1195637f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:31.110671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54629140-867c-4a75-8eb8-9f5b1195637f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:31.110671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54629140-867c-4a75-8eb8-9f5b1195637f admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.110671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54629140-867c-4a75-8eb8-9f5b1195637f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:31.111391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54629140-867c-4a75-8eb8-9f5b1195637f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:31.128093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1115/2245] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:31.128866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54629140-867c-4a75-8eb8-9f5b1195637f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:31.130561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-54629140-867c-4a75-8eb8-9f5b1195637f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:31.138940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3bd79c09-014a-4068-b020-d522c217a359 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:31.140390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3bd79c09-014a-4068-b020-d522c217a359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:31.149200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1131/2246] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:31.156823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd79c09-014a-4068-b020-d522c217a359 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:31.161628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3bd79c09-014a-4068-b020-d522c217a359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:31.173346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd79c09-014a-4068-b020-d522c217a359 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:31.177419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bc9b8-9ab0-4fcd-af6d-c23a2b08f786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:31.178075 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bc9b8-9ab0-4fcd-af6d-c23a2b08f786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:31.178482 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bc9b8-9ab0-4fcd-af6d-c23a2b08f786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:31.178746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bc9b8-9ab0-4fcd-af6d-c23a2b08f786 admin admin] PATH_INFO: `/v3/projects/564ce5a26c6f4469837611f2c9624d35/users/078673914b834c3096acd9cfb74a23f5/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.194486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bc9b8-9ab0-4fcd-af6d-c23a2b08f786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564ce5a26c6f4469837611f2c9624d35, user_id=078673914b834c3096acd9cfb74a23f5, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:31.197220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bc9b8-9ab0-4fcd-af6d-c23a2b08f786 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:31.198979 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54629140-867c-4a75-8eb8-9f5b1195637f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:31.202966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f41ac-19e7-45ed-8b67-6b15437b457d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:31.203641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f41ac-19e7-45ed-8b67-6b15437b457d service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:31.203989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f41ac-19e7-45ed-8b67-6b15437b457d service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:31.204355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f41ac-19e7-45ed-8b67-6b15437b457d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.213760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f41ac-19e7-45ed-8b67-6b15437b457d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:31.215176 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f41ac-19e7-45ed-8b67-6b15437b457d service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:31.224522 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1116/2247] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:31.225674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f41ac-19e7-45ed-8b67-6b15437b457d service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:31.227457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-261f41ac-19e7-45ed-8b67-6b15437b457d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:31.271276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:31.279852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1132/2248] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:31 2022] PUT /identity/v3/projects/564ce5a26c6f4469837611f2c9624d35/users/078673914b834c3096acd9cfb74a23f5/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:31.283761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bc9b8-9ab0-4fcd-af6d-c23a2b08f786 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:31.290949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f41ac-19e7-45ed-8b67-6b15437b457d service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:31.343302 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bb803-5757-4c00-b4a3-3e38c369b8c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:31.343997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bb803-5757-4c00-b4a3-3e38c369b8c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:31.344361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bb803-5757-4c00-b4a3-3e38c369b8c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:31.344712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bb803-5757-4c00-b4a3-3e38c369b8c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.354473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bc9b8-9ab0-4fcd-af6d-c23a2b08f786 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:31.356274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bb803-5757-4c00-b4a3-3e38c369b8c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:31.357827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bb803-5757-4c00-b4a3-3e38c369b8c0 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:31.359894 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e661b77-45ed-428f-a690-b8bfb23b3441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:31.362477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e661b77-45ed-428f-a690-b8bfb23b3441 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:31.362477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e661b77-45ed-428f-a690-b8bfb23b3441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:31.362477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e661b77-45ed-428f-a690-b8bfb23b3441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.362477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e661b77-45ed-428f-a690-b8bfb23b3441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:31.364247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e661b77-45ed-428f-a690-b8bfb23b3441 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:31.369432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1117/2249] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:31.376833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1133/2250] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:31.385157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bb803-5757-4c00-b4a3-3e38c369b8c0 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:31.397368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bb803-5757-4c00-b4a3-3e38c369b8c0 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:31.403532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc33d8-a806-44cc-8150-246d5101058b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:31.404226 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc33d8-a806-44cc-8150-246d5101058b admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:31.404514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc33d8-a806-44cc-8150-246d5101058b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:31.404800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc33d8-a806-44cc-8150-246d5101058b admin admin] PATH_INFO: `/v3/projects/564ce5a26c6f4469837611f2c9624d35/users/078673914b834c3096acd9cfb74a23f5/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.406799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc33d8-a806-44cc-8150-246d5101058b admin admin] RBAC: Authorizing `identity:create_grant(project_id=564ce5a26c6f4469837611f2c9624d35, user_id=078673914b834c3096acd9cfb74a23f5, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:31.410280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc33d8-a806-44cc-8150-246d5101058b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:31.434046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1118/2251] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:31 2022] PUT /identity/v3/projects/564ce5a26c6f4469837611f2c9624d35/users/078673914b834c3096acd9cfb74a23f5/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:31.435986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d668d2-1d7a-4898-9294-29607d5ce15b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:31.436298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d668d2-1d7a-4898-9294-29607d5ce15b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:31.436575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d668d2-1d7a-4898-9294-29607d5ce15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.465741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-52d668d2-1d7a-4898-9294-29607d5ce15b None None] MFA Rules not processed for user `078673914b834c3096acd9cfb74a23f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:31.467117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-52d668d2-1d7a-4898-9294-29607d5ce15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:31.541266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1134/2252] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:31.552490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc33d8-a806-44cc-8150-246d5101058b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:31.554444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-94cc33d8-a806-44cc-8150-246d5101058b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:31.614550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc33d8-a806-44cc-8150-246d5101058b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:31.621327 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b0a84-3abd-4f09-973b-926dc1304131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:31.623344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b0a84-3abd-4f09-973b-926dc1304131 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:31.623344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b0a84-3abd-4f09-973b-926dc1304131 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:31.623344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b0a84-3abd-4f09-973b-926dc1304131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.635568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b0a84-3abd-4f09-973b-926dc1304131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:31.637285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b0a84-3abd-4f09-973b-926dc1304131 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:31.647773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1119/2253] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:31.755063 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d668d2-1d7a-4898-9294-29607d5ce15b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:31.828487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d668d2-1d7a-4898-9294-29607d5ce15b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:31.839800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59026b6-4e08-43b8-9f63-c00e65781ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:31.840562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59026b6-4e08-43b8-9f63-c00e65781ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:31.840862 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59026b6-4e08-43b8-9f63-c00e65781ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:31.841159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59026b6-4e08-43b8-9f63-c00e65781ae2 admin admin] PATH_INFO: `/v3/users/1084064a53aa4dba9b14b23a3f485298` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.847247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59026b6-4e08-43b8-9f63-c00e65781ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1084064a53aa4dba9b14b23a3f485298)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:31.847247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:31.847637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59026b6-4e08-43b8-9f63-c00e65781ae2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:31.945229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b0a84-3abd-4f09-973b-926dc1304131 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:31.945229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f59b0a84-3abd-4f09-973b-926dc1304131 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:31.957848 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b0a84-3abd-4f09-973b-926dc1304131 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:31.965251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:31.968310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:31.968310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:31.968310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] PATH_INFO: `/v3/projects/533d60c92a8d450481f89bbeed2e97b5` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:31.968310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=533d60c92a8d450481f89bbeed2e97b5)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:31.970036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:32.043323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b59026b6-4e08-43b8-9f63-c00e65781ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1084064a53aa4dba9b14b23a3f485298'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:32.078364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '533d60c92a8d450481f89bbeed2e97b5', 'request_id': 'req-d8221542-c49a-4578-89d5-b64f85d5e936'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:32.084906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '533d60c92a8d450481f89bbeed2e97b5', 'request_id': 'req-d8221542-c49a-4578-89d5-b64f85d5e936'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:32.146404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b59026b6-4e08-43b8-9f63-c00e65781ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1084064a53aa4dba9b14b23a3f485298'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:32.174748 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b59026b6-4e08-43b8-9f63-c00e65781ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1084064a53aa4dba9b14b23a3f485298'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:32.182510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b59026b6-4e08-43b8-9f63-c00e65781ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1084064a53aa4dba9b14b23a3f485298'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:32.183463 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b59026b6-4e08-43b8-9f63-c00e65781ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1084064a53aa4dba9b14b23a3f485298'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:32.189532 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1135/2254] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:31 2022] DELETE /identity/v3/users/1084064a53aa4dba9b14b23a3f485298 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:32.202136 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2d07-6fa7-47a9-98ed-0e542b2d4f7b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:32.202449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2d07-6fa7-47a9-98ed-0e542b2d4f7b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:32.202738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2d07-6fa7-47a9-98ed-0e542b2d4f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:32.208960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '533d60c92a8d450481f89bbeed2e97b5', 'request_id': 'req-d8221542-c49a-4578-89d5-b64f85d5e936'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:32.228174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8d2e2d07-6fa7-47a9-98ed-0e542b2d4f7b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:32.229915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] The token cache is being invalidate because project 533d60c92a8d450481f89bbeed2e97b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:32.230219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:32.230897 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d8221542-c49a-4578-89d5-b64f85d5e936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:32.231163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:32.232507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8d2e2d07-6fa7-47a9-98ed-0e542b2d4f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:32.233798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1120/2255] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:31 2022] DELETE /identity/v3/projects/533d60c92a8d450481f89bbeed2e97b5 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:32.238999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ee0c4-5071-479b-8d33-b4474f6d0dcf None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:32.239698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ee0c4-5071-479b-8d33-b4474f6d0dcf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:32.240928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ee0c4-5071-479b-8d33-b4474f6d0dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:32.264034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f07ee0c4-5071-479b-8d33-b4474f6d0dcf None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:32.265452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f07ee0c4-5071-479b-8d33-b4474f6d0dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:32.271088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:32.304992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1136/2256] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:32.311696 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e2d07-6fa7-47a9-98ed-0e542b2d4f7b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:32.313310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8d2e2d07-6fa7-47a9-98ed-0e542b2d4f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:32.332534 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1121/2257] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:32.346119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0b592-9ed2-4f3d-83d5-0b3e0bcd3f62 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:32.346675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0b592-9ed2-4f3d-83d5-0b3e0bcd3f62 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:32.347177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0b592-9ed2-4f3d-83d5-0b3e0bcd3f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:32.375583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e4b0b592-9ed2-4f3d-83d5-0b3e0bcd3f62 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:32.376905 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e4b0b592-9ed2-4f3d-83d5-0b3e0bcd3f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:32.386437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1122/2258] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:32.387359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e2d07-6fa7-47a9-98ed-0e542b2d4f7b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:32.390597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0b592-9ed2-4f3d-83d5-0b3e0bcd3f62 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:32.396508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168dbfe-2c12-4700-9d0d-2f8086d160ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:32.396508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168dbfe-2c12-4700-9d0d-2f8086d160ca service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:32.396508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168dbfe-2c12-4700-9d0d-2f8086d160ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:32.396508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168dbfe-2c12-4700-9d0d-2f8086d160ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:32.404186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0b592-9ed2-4f3d-83d5-0b3e0bcd3f62 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:32.408568 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168dbfe-2c12-4700-9d0d-2f8086d160ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:32.409101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f0590-3557-49f3-82fe-78a384bdd797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:32.409705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f0590-3557-49f3-82fe-78a384bdd797 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:32.409988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f0590-3557-49f3-82fe-78a384bdd797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:32.410115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168dbfe-2c12-4700-9d0d-2f8086d160ca service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:32.410256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f0590-3557-49f3-82fe-78a384bdd797 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:32.410907 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f0590-3557-49f3-82fe-78a384bdd797 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:32.411213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f0590-3557-49f3-82fe-78a384bdd797 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:32.412870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f0590-3557-49f3-82fe-78a384bdd797 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:32.429058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1123/2259] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:39:32.431015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f0590-3557-49f3-82fe-78a384bdd797 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:32.433506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1137/2260] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:32.441832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f0590-3557-49f3-82fe-78a384bdd797 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:32.447548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed7afd-429f-4c48-b75e-26b47270b4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:32.448193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed7afd-429f-4c48-b75e-26b47270b4cc admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:32.448468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed7afd-429f-4c48-b75e-26b47270b4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:32.448841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed7afd-429f-4c48-b75e-26b47270b4cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:32.449369 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed7afd-429f-4c48-b75e-26b47270b4cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:32.451256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed7afd-429f-4c48-b75e-26b47270b4cc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:32.539387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1124/2261] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:32 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:32.540757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168dbfe-2c12-4700-9d0d-2f8086d160ca service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:32.550600 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168dbfe-2c12-4700-9d0d-2f8086d160ca service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:32.609560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26d961-4b32-4c83-8a33-93404c1195f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:32.610178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26d961-4b32-4c83-8a33-93404c1195f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:32.610463 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26d961-4b32-4c83-8a33-93404c1195f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:32.610738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26d961-4b32-4c83-8a33-93404c1195f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:32.611415 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26d961-4b32-4c83-8a33-93404c1195f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:32.613434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26d961-4b32-4c83-8a33-93404c1195f8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:32.677814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:32.709668 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1138/2262] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:32 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:32.711917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed7afd-429f-4c48-b75e-26b47270b4cc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:32.730441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed7afd-429f-4c48-b75e-26b47270b4cc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:32.735877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d51d5-72a4-442f-9f30-818f63a78c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:32.736852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d51d5-72a4-442f-9f30-818f63a78c72 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:32.737138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d51d5-72a4-442f-9f30-818f63a78c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:32.737714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d51d5-72a4-442f-9f30-818f63a78c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:32.738428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d51d5-72a4-442f-9f30-818f63a78c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:32.740408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d51d5-72a4-442f-9f30-818f63a78c72 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:32.756547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1125/2263] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:32.761358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26d961-4b32-4c83-8a33-93404c1195f8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:32.772658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26d961-4b32-4c83-8a33-93404c1195f8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:32.778977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f58dc-1a3a-46d1-9a36-bd2f37193e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:32.779686 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f58dc-1a3a-46d1-9a36-bd2f37193e06 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:32.780141 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f58dc-1a3a-46d1-9a36-bd2f37193e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:32.780524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f58dc-1a3a-46d1-9a36-bd2f37193e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:32.781732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f58dc-1a3a-46d1-9a36-bd2f37193e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:32.783735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f58dc-1a3a-46d1-9a36-bd2f37193e06 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:32.794902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1139/2264] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:32.799220 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d51d5-72a4-442f-9f30-818f63a78c72 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:32.809827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d51d5-72a4-442f-9f30-818f63a78c72 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:32.814436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ae7e3-ab04-49ee-aebd-d1516c1241d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:32.815077 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ae7e3-ab04-49ee-aebd-d1516c1241d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:32.815349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ae7e3-ab04-49ee-aebd-d1516c1241d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:32.815618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ae7e3-ab04-49ee-aebd-d1516c1241d4 admin admin] PATH_INFO: `/v3/projects/453efae9703a4c7c91c71faf40e3e8c5/users/cc0a69532c9248c98fdbc83f0698610c/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:32.831505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ae7e3-ab04-49ee-aebd-d1516c1241d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=453efae9703a4c7c91c71faf40e3e8c5, user_id=cc0a69532c9248c98fdbc83f0698610c, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:32.834177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ae7e3-ab04-49ee-aebd-d1516c1241d4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:32.884119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:32.906840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f58dc-1a3a-46d1-9a36-bd2f37193e06 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:32.919852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1126/2265] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:32 2022] PUT /identity/v3/projects/453efae9703a4c7c91c71faf40e3e8c5/users/cc0a69532c9248c98fdbc83f0698610c/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:32.919852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ae7e3-ab04-49ee-aebd-d1516c1241d4 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:32.919852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7c0ae7e3-ab04-49ee-aebd-d1516c1241d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:32.922529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9f2f58dc-1a3a-46d1-9a36-bd2f37193e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:32.942420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f58dc-1a3a-46d1-9a36-bd2f37193e06 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:33.009292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ae7e3-ab04-49ee-aebd-d1516c1241d4 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:33.014568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:33.015205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:33.015486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:33.015760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] PATH_INFO: `/v3/projects/3ec4880f316c4710a5aab05ac78a1836` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:33.017064 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec4880f316c4710a5aab05ac78a1836)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:33.019148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:33.022597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:33.023347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:33.024274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:33.028240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] PATH_INFO: `/v3/users/509d6620f7bc42c798ea20d9e88917f1` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:33.028240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=509d6620f7bc42c798ea20d9e88917f1)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:33.028240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:33.178486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:33.187835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec4880f316c4710a5aab05ac78a1836', 'request_id': 'req-c76fb276-e906-4126-b0a9-71f11b5869d1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.193837 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec4880f316c4710a5aab05ac78a1836', 'request_id': 'req-c76fb276-e906-4126-b0a9-71f11b5869d1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.261768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec4880f316c4710a5aab05ac78a1836', 'request_id': 'req-c76fb276-e906-4126-b0a9-71f11b5869d1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.280467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '509d6620f7bc42c798ea20d9e88917f1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.281211 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] The token cache is being invalidate because project 3ec4880f316c4710a5aab05ac78a1836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:33.281509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.282174 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c76fb276-e906-4126-b0a9-71f11b5869d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.287007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1127/2266] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:32 2022] DELETE /identity/v3/projects/3ec4880f316c4710a5aab05ac78a1836 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:33.288336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d01ae-5163-471a-980b-7f39f1a3fc08 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:33.288620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d01ae-5163-471a-980b-7f39f1a3fc08 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:33.288900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d01ae-5163-471a-980b-7f39f1a3fc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:33.326173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7f3d01ae-5163-471a-980b-7f39f1a3fc08 None None] MFA Rules not processed for user `cc0a69532c9248c98fdbc83f0698610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:33.330222 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7f3d01ae-5163-471a-980b-7f39f1a3fc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:33.379975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '509d6620f7bc42c798ea20d9e88917f1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.395429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '509d6620f7bc42c798ea20d9e88917f1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.399893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1128/2267] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:33.400926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d01ae-5163-471a-980b-7f39f1a3fc08 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:33.402497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7f3d01ae-5163-471a-980b-7f39f1a3fc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:33.407330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '509d6620f7bc42c798ea20d9e88917f1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.409607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '509d6620f7bc42c798ea20d9e88917f1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.417021 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1140/2268] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:32 2022] DELETE /identity/v3/users/509d6620f7bc42c798ea20d9e88917f1 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:33.420123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:33.420123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:33.470465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d01ae-5163-471a-980b-7f39f1a3fc08 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:33.476484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea218bf5-6da7-417a-a2d6-3dceea3c2a80 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:33.480386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4f2ea-d1ee-4ae8-ba4f-68b658dc1f22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:33.480996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4f2ea-d1ee-4ae8-ba4f-68b658dc1f22 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:33.481274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4f2ea-d1ee-4ae8-ba4f-68b658dc1f22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:33.481543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4f2ea-d1ee-4ae8-ba4f-68b658dc1f22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:33.483321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f5c4f2ea-d1ee-4ae8-ba4f-68b658dc1f22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:33.494311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:33.527104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:33.527747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:33.528315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:33.528750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] PATH_INFO: `/v3/users/078673914b834c3096acd9cfb74a23f5` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:33.530362 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=078673914b834c3096acd9cfb74a23f5)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:33.532461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:33.547575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4f2ea-d1ee-4ae8-ba4f-68b658dc1f22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:33.549051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4f2ea-d1ee-4ae8-ba4f-68b658dc1f22 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:33.558970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1141/2269] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:33.560247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e19fbc-e782-46d2-8bca-61383f975b9f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:33.560529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e19fbc-e782-46d2-8bca-61383f975b9f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:33.560802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e19fbc-e782-46d2-8bca-61383f975b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:33.583007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b2e19fbc-e782-46d2-8bca-61383f975b9f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:33.583007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b2e19fbc-e782-46d2-8bca-61383f975b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:33.588721 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1142/2270] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:33.589523 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e19fbc-e782-46d2-8bca-61383f975b9f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:33.590820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b2e19fbc-e782-46d2-8bca-61383f975b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:33.603363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e19fbc-e782-46d2-8bca-61383f975b9f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:33.609451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4487ee3-0636-430b-b7ff-a32439431ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:33.610470 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4487ee3-0636-430b-b7ff-a32439431ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:33.610864 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4487ee3-0636-430b-b7ff-a32439431ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:33.611111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4487ee3-0636-430b-b7ff-a32439431ef1 admin admin] PATH_INFO: `/v3/users/cc0a69532c9248c98fdbc83f0698610c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:33.612258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4487ee3-0636-430b-b7ff-a32439431ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc0a69532c9248c98fdbc83f0698610c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:33.614466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4487ee3-0636-430b-b7ff-a32439431ef1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:33.769928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '078673914b834c3096acd9cfb74a23f5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.861181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:33.863593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '078673914b834c3096acd9cfb74a23f5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.880909 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '078673914b834c3096acd9cfb74a23f5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.881076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f4487ee3-0636-430b-b7ff-a32439431ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc0a69532c9248c98fdbc83f0698610c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.882089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '078673914b834c3096acd9cfb74a23f5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.883239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '078673914b834c3096acd9cfb74a23f5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:33.897954 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1129/2271] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:33 2022] DELETE /identity/v3/users/078673914b834c3096acd9cfb74a23f5 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:33.901294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:33.901294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:33.976285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce8da3-4add-46f8-b4a1-160793d71069 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:33.988693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7cd77-94da-480b-9e51-6171bd318171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:33.989493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7cd77-94da-480b-9e51-6171bd318171 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:33.989933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7cd77-94da-480b-9e51-6171bd318171 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:33.990305 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7cd77-94da-480b-9e51-6171bd318171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:33.994636 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7bf7cd77-94da-480b-9e51-6171bd318171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:33.998733 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f4487ee3-0636-430b-b7ff-a32439431ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc0a69532c9248c98fdbc83f0698610c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:34.015387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f4487ee3-0636-430b-b7ff-a32439431ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc0a69532c9248c98fdbc83f0698610c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:34.024525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f4487ee3-0636-430b-b7ff-a32439431ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc0a69532c9248c98fdbc83f0698610c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:34.025472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f4487ee3-0636-430b-b7ff-a32439431ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc0a69532c9248c98fdbc83f0698610c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:34.031607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1143/2272] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:33 2022] DELETE /identity/v3/users/cc0a69532c9248c98fdbc83f0698610c => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:34.038112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01027987-ac27-477c-87dd-2e2d2162319e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:34.038403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01027987-ac27-477c-87dd-2e2d2162319e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:34.039384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01027987-ac27-477c-87dd-2e2d2162319e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:34.064941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-01027987-ac27-477c-87dd-2e2d2162319e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:34.065860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-01027987-ac27-477c-87dd-2e2d2162319e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:34.090557 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7cd77-94da-480b-9e51-6171bd318171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:34.093177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7cd77-94da-480b-9e51-6171bd318171 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:34.128069 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1130/2273] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:34.157791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1144/2274] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:34.170542 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7cd77-94da-480b-9e51-6171bd318171 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:34.179498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7cd77-94da-480b-9e51-6171bd318171 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:34.242240 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d370e6-c319-4700-b941-6655e308bc39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:34.243500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d370e6-c319-4700-b941-6655e308bc39 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:34.243774 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d370e6-c319-4700-b941-6655e308bc39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:34.244101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d370e6-c319-4700-b941-6655e308bc39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:34.254052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d370e6-c319-4700-b941-6655e308bc39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:34.256667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d370e6-c319-4700-b941-6655e308bc39 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:34.266414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1131/2275] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:39:34.687186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01027987-ac27-477c-87dd-2e2d2162319e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:34.689200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-01027987-ac27-477c-87dd-2e2d2162319e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:34.710691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01027987-ac27-477c-87dd-2e2d2162319e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:34.720752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:34.721692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:34.721692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:34.722570 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] PATH_INFO: `/v3/projects/f8b3b659d6b0455ebf5447e5563c3e15` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:34.726295 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b3b659d6b0455ebf5447e5563c3e15)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:34.741496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:34.766893 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:34.840180 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d370e6-c319-4700-b941-6655e308bc39 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:34.841626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-91d370e6-c319-4700-b941-6655e308bc39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:34.870311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d370e6-c319-4700-b941-6655e308bc39 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:34.887695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b3b659d6b0455ebf5447e5563c3e15', 'request_id': 'req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:34.897788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b3b659d6b0455ebf5447e5563c3e15', 'request_id': 'req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:34.949501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:34.950238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:34.950587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:34.950945 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] PATH_INFO: `/v3/projects/453efae9703a4c7c91c71faf40e3e8c5` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:34.952286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=453efae9703a4c7c91c71faf40e3e8c5)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:34.952395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b3b659d6b0455ebf5447e5563c3e15', 'request_id': 'req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:34.954407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:34.967731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] The token cache is being invalidate because project f8b3b659d6b0455ebf5447e5563c3e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:34.968340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:34.969282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:34.971221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1145/2276] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:34 2022] DELETE /identity/v3/projects/f8b3b659d6b0455ebf5447e5563c3e15 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:34.972026 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:34.973374 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:35.003910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2832a-568d-4a72-aa6d-3c2d9d073b8e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:35.013275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:35.013680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:35.013824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:35.014131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] PATH_INFO: `/v3/projects/564ce5a26c6f4469837611f2c9624d35` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:35.015326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=564ce5a26c6f4469837611f2c9624d35)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:35.017366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:35.077692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453efae9703a4c7c91c71faf40e3e8c5', 'request_id': 'req-94ed339f-b40c-4c0c-8149-7c024ae06c27'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.084407 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453efae9703a4c7c91c71faf40e3e8c5', 'request_id': 'req-94ed339f-b40c-4c0c-8149-7c024ae06c27'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.191564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453efae9703a4c7c91c71faf40e3e8c5', 'request_id': 'req-94ed339f-b40c-4c0c-8149-7c024ae06c27'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.196500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '564ce5a26c6f4469837611f2c9624d35', 'request_id': 'req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.200402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '564ce5a26c6f4469837611f2c9624d35', 'request_id': 'req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.208466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] The token cache is being invalidate because project 453efae9703a4c7c91c71faf40e3e8c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:35.208757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.209569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.211648 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1132/2277] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:34 2022] DELETE /identity/v3/projects/453efae9703a4c7c91c71faf40e3e8c5 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:35.215001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:35.215001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:35.281255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed339f-b40c-4c0c-8149-7c024ae06c27 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:35.288805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '564ce5a26c6f4469837611f2c9624d35', 'request_id': 'req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.300954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] The token cache is being invalidate because project 564ce5a26c6f4469837611f2c9624d35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:35.301384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.302053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89ee7eb1-5783-40c2-a1aa-7a7d59c90612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.303874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1146/2278] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:34 2022] DELETE /identity/v3/projects/564ce5a26c6f4469837611f2c9624d35 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:35.305704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cbfd0-724f-4662-a76f-df3937f01692 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:35.305985 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cbfd0-724f-4662-a76f-df3937f01692 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:35.306254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cbfd0-724f-4662-a76f-df3937f01692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:35.327273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7f5cbfd0-724f-4662-a76f-df3937f01692 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:35.333755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7f5cbfd0-724f-4662-a76f-df3937f01692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:35.351527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:35.353391 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:35.353885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:35.354238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:35.356279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] PATH_INFO: `/v3/users/dccab8e799244d6ebf9076fe23a2c5e9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:35.356279 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dccab8e799244d6ebf9076fe23a2c5e9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:35.357970 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:35.392304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1147/2279] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:35.393584 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcba806-d4bf-41ab-85ed-10fa83b30de0 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:35.393966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcba806-d4bf-41ab-85ed-10fa83b30de0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:35.394265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcba806-d4bf-41ab-85ed-10fa83b30de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:35.413802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bdcba806-d4bf-41ab-85ed-10fa83b30de0 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:35.415166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bdcba806-d4bf-41ab-85ed-10fa83b30de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:35.424383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1148/2280] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:35.431799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefda14-ec24-497c-bcf9-002448742899 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:35.431799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefda14-ec24-497c-bcf9-002448742899 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:35.431799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefda14-ec24-497c-bcf9-002448742899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:35.449046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8aefda14-ec24-497c-bcf9-002448742899 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:35.450257 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8aefda14-ec24-497c-bcf9-002448742899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:35.458228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1149/2281] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:35.459459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c26e3-506c-438e-a1d3-b9216c6613bc None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:35.459737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c26e3-506c-438e-a1d3-b9216c6613bc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:35.460006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c26e3-506c-438e-a1d3-b9216c6613bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:35.478988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-485c26e3-506c-438e-a1d3-b9216c6613bc None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:35.480183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-485c26e3-506c-438e-a1d3-b9216c6613bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:35.488263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1150/2282] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:35.489015 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c26e3-506c-438e-a1d3-b9216c6613bc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:35.499648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c26e3-506c-438e-a1d3-b9216c6613bc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:35.504263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52473f-5bf7-42c5-94fd-ac56743d47cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:35.504906 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52473f-5bf7-42c5-94fd-ac56743d47cb admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:35.505180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52473f-5bf7-42c5-94fd-ac56743d47cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:35.505450 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52473f-5bf7-42c5-94fd-ac56743d47cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:35.506093 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52473f-5bf7-42c5-94fd-ac56743d47cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:35.506395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52473f-5bf7-42c5-94fd-ac56743d47cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:35.508848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52473f-5bf7-42c5-94fd-ac56743d47cb admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:35.521537 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1151/2283] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:35.522299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52473f-5bf7-42c5-94fd-ac56743d47cb admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:35.532168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52473f-5bf7-42c5-94fd-ac56743d47cb admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:35.534395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dccab8e799244d6ebf9076fe23a2c5e9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.536904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed44d80-46f2-4079-b491-04571d52a10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:35.540094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed44d80-46f2-4079-b491-04571d52a10e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:35.540094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed44d80-46f2-4079-b491-04571d52a10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:35.540094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed44d80-46f2-4079-b491-04571d52a10e admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:35.540094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed44d80-46f2-4079-b491-04571d52a10e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:35.540094 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed44d80-46f2-4079-b491-04571d52a10e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:35.540648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed44d80-46f2-4079-b491-04571d52a10e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:35.552730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1152/2284] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:35.553531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed44d80-46f2-4079-b491-04571d52a10e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:35.564304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed44d80-46f2-4079-b491-04571d52a10e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:35.569426 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924e80c-174a-48b4-977d-8a42200a5375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:35.570023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e80c-174a-48b4-977d-8a42200a5375 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:35.570921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e80c-174a-48b4-977d-8a42200a5375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:35.571375 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e80c-174a-48b4-977d-8a42200a5375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:35.572050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924e80c-174a-48b4-977d-8a42200a5375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:35.584178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924e80c-174a-48b4-977d-8a42200a5375 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:35.614510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dccab8e799244d6ebf9076fe23a2c5e9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.633923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dccab8e799244d6ebf9076fe23a2c5e9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.633923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dccab8e799244d6ebf9076fe23a2c5e9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.633923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dccab8e799244d6ebf9076fe23a2c5e9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:35.642610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1133/2285] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:35 2022] DELETE /identity/v3/users/dccab8e799244d6ebf9076fe23a2c5e9 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:35.643441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:35.644862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:35.718915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1153/2286] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:35 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:35.719790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924e80c-174a-48b4-977d-8a42200a5375 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:35.722673 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0924e80c-174a-48b4-977d-8a42200a5375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:35.737151 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ecfdb-63e5-49c3-a416-5212cbfa6f06 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:35.793660 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda48c2-dc84-444c-b2b0-b2001c230484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:35.794293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda48c2-dc84-444c-b2b0-b2001c230484 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:35.794585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda48c2-dc84-444c-b2b0-b2001c230484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:35.795121 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda48c2-dc84-444c-b2b0-b2001c230484 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:35.795771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda48c2-dc84-444c-b2b0-b2001c230484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:35.797720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda48c2-dc84-444c-b2b0-b2001c230484 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:35.850127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924e80c-174a-48b4-977d-8a42200a5375 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:35.856640 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395771bf-e82f-4836-974a-0748ace363e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:35.857387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395771bf-e82f-4836-974a-0748ace363e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:35.857779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395771bf-e82f-4836-974a-0748ace363e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:35.858148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395771bf-e82f-4836-974a-0748ace363e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:35.859513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395771bf-e82f-4836-974a-0748ace363e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:35.861173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395771bf-e82f-4836-974a-0748ace363e9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:35.903874 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1134/2287] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:35 2022] POST /identity/v3/projects => generated 391 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:35.905784 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda48c2-dc84-444c-b2b0-b2001c230484 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:35.949526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1154/2288] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:35 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:35.965634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395771bf-e82f-4836-974a-0748ace363e9 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:35.985373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395771bf-e82f-4836-974a-0748ace363e9 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.018387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda48c2-dc84-444c-b2b0-b2001c230484 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.024934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2420ae8-23df-451d-8972-10b4a31775b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.026452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2420ae8-23df-451d-8972-10b4a31775b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.026790 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2420ae8-23df-451d-8972-10b4a31775b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.027062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2420ae8-23df-451d-8972-10b4a31775b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.027839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2420ae8-23df-451d-8972-10b4a31775b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.029906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2420ae8-23df-451d-8972-10b4a31775b0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.069422 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e263508-13ad-43b6-9c9d-80d9c19e0664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.071004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e263508-13ad-43b6-9c9d-80d9c19e0664 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.071004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e263508-13ad-43b6-9c9d-80d9c19e0664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.071004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e263508-13ad-43b6-9c9d-80d9c19e0664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.071528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e263508-13ad-43b6-9c9d-80d9c19e0664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.073160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e263508-13ad-43b6-9c9d-80d9c19e0664 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.082968 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1155/2289] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:36.087904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e263508-13ad-43b6-9c9d-80d9c19e0664 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.098753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e263508-13ad-43b6-9c9d-80d9c19e0664 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.103896 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536298b6-2e19-4a93-ac73-7b409fbb7e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.104556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536298b6-2e19-4a93-ac73-7b409fbb7e15 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.104834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536298b6-2e19-4a93-ac73-7b409fbb7e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.105124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536298b6-2e19-4a93-ac73-7b409fbb7e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.105749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536298b6-2e19-4a93-ac73-7b409fbb7e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.107402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536298b6-2e19-4a93-ac73-7b409fbb7e15 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.117333 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1156/2290] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:36.121667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536298b6-2e19-4a93-ac73-7b409fbb7e15 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.131548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536298b6-2e19-4a93-ac73-7b409fbb7e15 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.137452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e703ed-6074-4a82-a5f8-e05d1a89feed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.138097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e703ed-6074-4a82-a5f8-e05d1a89feed admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.138376 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e703ed-6074-4a82-a5f8-e05d1a89feed admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.138651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e703ed-6074-4a82-a5f8-e05d1a89feed admin admin] PATH_INFO: `/v3/projects/5eea609cee794549a2c23cdb24fbb572/users/def1da13ff2a47ada8e5f453061e3f7c/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.140925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1135/2291] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:35 2022] POST /identity/v3/users => generated 296 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:36.144932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2420ae8-23df-451d-8972-10b4a31775b0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.155795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e703ed-6074-4a82-a5f8-e05d1a89feed admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eea609cee794549a2c23cdb24fbb572, user_id=def1da13ff2a47ada8e5f453061e3f7c, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.156233 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2420ae8-23df-451d-8972-10b4a31775b0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.158708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e703ed-6074-4a82-a5f8-e05d1a89feed admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.161405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a99a-352e-415b-9b79-83fdf2b88de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.161996 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a99a-352e-415b-9b79-83fdf2b88de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.162274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a99a-352e-415b-9b79-83fdf2b88de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.162560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a99a-352e-415b-9b79-83fdf2b88de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.163190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7a99a-352e-415b-9b79-83fdf2b88de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.165889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7a99a-352e-415b-9b79-83fdf2b88de5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.179742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1136/2292] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:36.179742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a99a-352e-415b-9b79-83fdf2b88de5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.179742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b2d7a99a-352e-415b-9b79-83fdf2b88de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:36.196641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a99a-352e-415b-9b79-83fdf2b88de5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.201708 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.202257 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.202544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.202823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] PATH_INFO: `/v3/projects/0fe0b5ec1aad4bc5b5bea74b8fd23852` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.204336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe0b5ec1aad4bc5b5bea74b8fd23852)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.206286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.225722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1157/2293] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:36 2022] PUT /identity/v3/projects/5eea609cee794549a2c23cdb24fbb572/users/def1da13ff2a47ada8e5f453061e3f7c/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:36.227196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e703ed-6074-4a82-a5f8-e05d1a89feed admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.291319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e703ed-6074-4a82-a5f8-e05d1a89feed admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.296920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39588e4d-40b6-41fa-a951-9ffee49c98d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.297480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39588e4d-40b6-41fa-a951-9ffee49c98d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.297795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39588e4d-40b6-41fa-a951-9ffee49c98d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.298062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39588e4d-40b6-41fa-a951-9ffee49c98d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.298780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39588e4d-40b6-41fa-a951-9ffee49c98d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.300431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39588e4d-40b6-41fa-a951-9ffee49c98d6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.309061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe0b5ec1aad4bc5b5bea74b8fd23852', 'request_id': 'req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:36.317098 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1158/2294] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:36.318185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe0b5ec1aad4bc5b5bea74b8fd23852', 'request_id': 'req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:36.318409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6aa1b-ebfd-4f82-9713-a8f3f3990167 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.318639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6aa1b-ebfd-4f82-9713-a8f3f3990167 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.318920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6aa1b-ebfd-4f82-9713-a8f3f3990167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.340070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-dbd6aa1b-ebfd-4f82-9713-a8f3f3990167 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:36.341330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dbd6aa1b-ebfd-4f82-9713-a8f3f3990167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:36.392430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe0b5ec1aad4bc5b5bea74b8fd23852', 'request_id': 'req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:36.405535 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] The token cache is being invalidate because project 0fe0b5ec1aad4bc5b5bea74b8fd23852 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:36.405827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:36.406673 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:36.408561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1137/2295] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:36 2022] DELETE /identity/v3/projects/0fe0b5ec1aad4bc5b5bea74b8fd23852 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:36.409455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.410843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:36.415439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1159/2296] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:36.417695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c393f88-1e1f-4471-b05b-c827c4a56183 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.417695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c393f88-1e1f-4471-b05b-c827c4a56183 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.417695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c393f88-1e1f-4471-b05b-c827c4a56183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.432351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85bc74-d256-4b3e-a073-cfb9f65e16b1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.438355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa424d7-c431-49b1-a2a6-687c4c7378e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.439241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa424d7-c431-49b1-a2a6-687c4c7378e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.444421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa424d7-c431-49b1-a2a6-687c4c7378e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.444421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa424d7-c431-49b1-a2a6-687c4c7378e6 admin admin] PATH_INFO: `/v3/projects/d0b7aca06bfa4aabaf02b861161d04be/users/8faa3c70c6a543b58d71980e8f418310/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.444605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6c393f88-1e1f-4471-b05b-c827c4a56183 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:36.444605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6c393f88-1e1f-4471-b05b-c827c4a56183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:36.455145 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1160/2297] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:36.461751 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c393f88-1e1f-4471-b05b-c827c4a56183 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.463283 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6c393f88-1e1f-4471-b05b-c827c4a56183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:36.464697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa424d7-c431-49b1-a2a6-687c4c7378e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b7aca06bfa4aabaf02b861161d04be, user_id=8faa3c70c6a543b58d71980e8f418310, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.467493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa424d7-c431-49b1-a2a6-687c4c7378e6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.524171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c393f88-1e1f-4471-b05b-c827c4a56183 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.527938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a9edc-9d40-408d-8fa9-d044015125d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.528605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a9edc-9d40-408d-8fa9-d044015125d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.528888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a9edc-9d40-408d-8fa9-d044015125d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.529168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a9edc-9d40-408d-8fa9-d044015125d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.530956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c32a9edc-9d40-408d-8fa9-d044015125d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:36.543270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a9edc-9d40-408d-8fa9-d044015125d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.544790 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a9edc-9d40-408d-8fa9-d044015125d2 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.555146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1161/2298] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:36.556416 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbae5f3-2fa2-47e5-8caf-f82e0a560e04 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.556717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbae5f3-2fa2-47e5-8caf-f82e0a560e04 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.556965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbae5f3-2fa2-47e5-8caf-f82e0a560e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.567497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1138/2299] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:36 2022] PUT /identity/v3/projects/d0b7aca06bfa4aabaf02b861161d04be/users/8faa3c70c6a543b58d71980e8f418310/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:36.571326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c7c7a-acb7-4961-b10e-903c4897bada None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.571692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c7c7a-acb7-4961-b10e-903c4897bada None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.572232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c7c7a-acb7-4961-b10e-903c4897bada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.580926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:36.591965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-edbae5f3-2fa2-47e5-8caf-f82e0a560e04 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:36.593204 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-edbae5f3-2fa2-47e5-8caf-f82e0a560e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:36.600765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-541c7c7a-acb7-4961-b10e-903c4897bada None None] MFA Rules not processed for user `8faa3c70c6a543b58d71980e8f418310`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:36.602153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-541c7c7a-acb7-4961-b10e-903c4897bada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:36.664482 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1162/2300] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:36.666453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbae5f3-2fa2-47e5-8caf-f82e0a560e04 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.675738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1139/2301] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:36.680159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbae5f3-2fa2-47e5-8caf-f82e0a560e04 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.683999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c7c7a-acb7-4961-b10e-903c4897bada None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.686084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990be40e-de65-442c-9875-9fd94ce2ccad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.686710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990be40e-de65-442c-9875-9fd94ce2ccad admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.687005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990be40e-de65-442c-9875-9fd94ce2ccad admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.687293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990be40e-de65-442c-9875-9fd94ce2ccad admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.687948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990be40e-de65-442c-9875-9fd94ce2ccad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:36.688295 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990be40e-de65-442c-9875-9fd94ce2ccad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.689914 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990be40e-de65-442c-9875-9fd94ce2ccad admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.698163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c7c7a-acb7-4961-b10e-903c4897bada None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.716622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1163/2302] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:36.717086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29efa34-0c49-4fdb-ae32-ee648d31ba8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.717086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efa34-0c49-4fdb-ae32-ee648d31ba8b admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.717086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efa34-0c49-4fdb-ae32-ee648d31ba8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.717086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efa34-0c49-4fdb-ae32-ee648d31ba8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.717601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29efa34-0c49-4fdb-ae32-ee648d31ba8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.717601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29efa34-0c49-4fdb-ae32-ee648d31ba8b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.725319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990be40e-de65-442c-9875-9fd94ce2ccad admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.739334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990be40e-de65-442c-9875-9fd94ce2ccad admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.745948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288f24b-ccd6-4213-a37f-c64bdc6fad66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.746650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288f24b-ccd6-4213-a37f-c64bdc6fad66 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.746930 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288f24b-ccd6-4213-a37f-c64bdc6fad66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.747200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288f24b-ccd6-4213-a37f-c64bdc6fad66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.747833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288f24b-ccd6-4213-a37f-c64bdc6fad66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.749799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288f24b-ccd6-4213-a37f-c64bdc6fad66 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.800787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1140/2303] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:36 2022] POST /identity/v3/projects => generated 391 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:36.805772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29efa34-0c49-4fdb-ae32-ee648d31ba8b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.816150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1164/2304] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:36 2022] POST /identity/v3/projects => generated 387 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:36.820959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288f24b-ccd6-4213-a37f-c64bdc6fad66 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:36.911005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29efa34-0c49-4fdb-ae32-ee648d31ba8b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.917023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3ded6-e95b-46f3-bc4b-c17ddf587004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.918418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3ded6-e95b-46f3-bc4b-c17ddf587004 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.918796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3ded6-e95b-46f3-bc4b-c17ddf587004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.919133 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3ded6-e95b-46f3-bc4b-c17ddf587004 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.919927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3ded6-e95b-46f3-bc4b-c17ddf587004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.920606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288f24b-ccd6-4213-a37f-c64bdc6fad66 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:36.923576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3ded6-e95b-46f3-bc4b-c17ddf587004 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:36.931400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3eae6-c75f-4a20-90fd-980916e8efc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:36.932041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3eae6-c75f-4a20-90fd-980916e8efc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:36.946190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3eae6-c75f-4a20-90fd-980916e8efc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:36.952673 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3eae6-c75f-4a20-90fd-980916e8efc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:36.953592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3eae6-c75f-4a20-90fd-980916e8efc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:36.957123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3eae6-c75f-4a20-90fd-980916e8efc1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:37.006977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1141/2305] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:36 2022] POST /identity/v3/users => generated 302 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:37.009016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3ded6-e95b-46f3-bc4b-c17ddf587004 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:37.020080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3ded6-e95b-46f3-bc4b-c17ddf587004 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:37.027416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815f7a7-1787-4a9c-a6e3-5cafa39b632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:37.030781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:37.030781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815f7a7-1787-4a9c-a6e3-5cafa39b632d admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.030781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815f7a7-1787-4a9c-a6e3-5cafa39b632d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.030781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815f7a7-1787-4a9c-a6e3-5cafa39b632d admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:37.031518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a815f7a7-1787-4a9c-a6e3-5cafa39b632d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:37.033347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a815f7a7-1787-4a9c-a6e3-5cafa39b632d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:37.045397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1142/2306] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:37.049637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815f7a7-1787-4a9c-a6e3-5cafa39b632d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:37.061339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815f7a7-1787-4a9c-a6e3-5cafa39b632d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:37.069132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a744da5-dd46-45d9-a49a-0d10a02e7ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:37.076846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a744da5-dd46-45d9-a49a-0d10a02e7ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.076846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a744da5-dd46-45d9-a49a-0d10a02e7ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.076846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a744da5-dd46-45d9-a49a-0d10a02e7ad7 admin admin] PATH_INFO: `/v3/projects/ad763b4b13e246fe8bb4f3eddd016b2a/users/ec238b08150e4846821d5a1a8ae9addc/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:37.080110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1165/2307] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:36 2022] POST /identity/v3/users => generated 294 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:37.088587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3eae6-c75f-4a20-90fd-980916e8efc1 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:37.092603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a744da5-dd46-45d9-a49a-0d10a02e7ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad763b4b13e246fe8bb4f3eddd016b2a, user_id=ec238b08150e4846821d5a1a8ae9addc, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:37.095668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a744da5-dd46-45d9-a49a-0d10a02e7ad7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:37.160372 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3eae6-c75f-4a20-90fd-980916e8efc1 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:37.167771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc876a-a4e0-4fe9-8e1e-10e28c59a985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:37.168522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc876a-a4e0-4fe9-8e1e-10e28c59a985 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.168809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc876a-a4e0-4fe9-8e1e-10e28c59a985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.169085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc876a-a4e0-4fe9-8e1e-10e28c59a985 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:37.170236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc876a-a4e0-4fe9-8e1e-10e28c59a985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:37.173404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc876a-a4e0-4fe9-8e1e-10e28c59a985 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:37.199907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1166/2309] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:37.201656 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1143/2309] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:37 2022] PUT /identity/v3/projects/ad763b4b13e246fe8bb4f3eddd016b2a/users/ec238b08150e4846821d5a1a8ae9addc/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:37.203005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc876a-a4e0-4fe9-8e1e-10e28c59a985 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:37.209919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a744da5-dd46-45d9-a49a-0d10a02e7ad7 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:37.231506 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc876a-a4e0-4fe9-8e1e-10e28c59a985 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:37.354489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a744da5-dd46-45d9-a49a-0d10a02e7ad7 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:37.357157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ec913-5056-4a08-bc45-5964a8ec0f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:37.361927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ec913-5056-4a08-bc45-5964a8ec0f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.361927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ec913-5056-4a08-bc45-5964a8ec0f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.361927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ec913-5056-4a08-bc45-5964a8ec0f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:37.361927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ec913-5056-4a08-bc45-5964a8ec0f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:37.361927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ec913-5056-4a08-bc45-5964a8ec0f3d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:37.369525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88acb8-0610-4017-b702-3cca74fefd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:37.370255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88acb8-0610-4017-b702-3cca74fefd07 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.370647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88acb8-0610-4017-b702-3cca74fefd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.371021 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88acb8-0610-4017-b702-3cca74fefd07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:37.371761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88acb8-0610-4017-b702-3cca74fefd07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:37.372763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1167/2310] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:37.377171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88acb8-0610-4017-b702-3cca74fefd07 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:37.377637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ec913-5056-4a08-bc45-5964a8ec0f3d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:37.394380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1144/2311] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:37.394589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ec913-5056-4a08-bc45-5964a8ec0f3d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:37.396378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88acb8-0610-4017-b702-3cca74fefd07 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:37.401438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04960437-6564-4ca7-942a-8faf0a888fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:37.402419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04960437-6564-4ca7-942a-8faf0a888fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.403130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04960437-6564-4ca7-942a-8faf0a888fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.403555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04960437-6564-4ca7-942a-8faf0a888fd9 admin admin] PATH_INFO: `/v3/projects/ad763b4b13e246fe8bb4f3eddd016b2a/users/ec238b08150e4846821d5a1a8ae9addc/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:37.407020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04960437-6564-4ca7-942a-8faf0a888fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad763b4b13e246fe8bb4f3eddd016b2a, user_id=ec238b08150e4846821d5a1a8ae9addc, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:37.408338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88acb8-0610-4017-b702-3cca74fefd07 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:37.410573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04960437-6564-4ca7-942a-8faf0a888fd9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:37.413626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f11cc3-5e30-4c64-a43a-dcd4e6b919ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:37.414450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f11cc3-5e30-4c64-a43a-dcd4e6b919ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.414737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f11cc3-5e30-4c64-a43a-dcd4e6b919ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.415032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f11cc3-5e30-4c64-a43a-dcd4e6b919ee admin admin] PATH_INFO: `/v3/projects/8f5876445c2541c8b0051da9eef4a090/users/d321c9d029ef4e9fb67c2a09a24d82e8/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:37.437168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1168/2312] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:37 2022] PUT /identity/v3/projects/ad763b4b13e246fe8bb4f3eddd016b2a/users/ec238b08150e4846821d5a1a8ae9addc/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:37.448564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13b147-3979-4670-881b-30e59ab345d3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.448564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13b147-3979-4670-881b-30e59ab345d3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.448877 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13b147-3979-4670-881b-30e59ab345d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:37.459414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f11cc3-5e30-4c64-a43a-dcd4e6b919ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5876445c2541c8b0051da9eef4a090, user_id=d321c9d029ef4e9fb67c2a09a24d82e8, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:37.470476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f11cc3-5e30-4c64-a43a-dcd4e6b919ee admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:37.487301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7b13b147-3979-4670-881b-30e59ab345d3 None None] MFA Rules not processed for user `ec238b08150e4846821d5a1a8ae9addc`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:37.488928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7b13b147-3979-4670-881b-30e59ab345d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:37.579124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1145/2313] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:37 2022] PUT /identity/v3/projects/8f5876445c2541c8b0051da9eef4a090/users/d321c9d029ef4e9fb67c2a09a24d82e8/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:37.590490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3d5c6-a1ed-41b6-a683-989b8a29be4f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.592141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3d5c6-a1ed-41b6-a683-989b8a29be4f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.592141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3d5c6-a1ed-41b6-a683-989b8a29be4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:37.605903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1169/2314] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:37 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:37.619626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13b147-3979-4670-881b-30e59ab345d3 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:37.621133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7b13b147-3979-4670-881b-30e59ab345d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:37.653592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d2e3d5c6-a1ed-41b6-a683-989b8a29be4f None None] MFA Rules not processed for user `d321c9d029ef4e9fb67c2a09a24d82e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:37.654906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d2e3d5c6-a1ed-41b6-a683-989b8a29be4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:37.731603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13b147-3979-4670-881b-30e59ab345d3 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:37.762336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1146/2315] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:37.764123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3d5c6-a1ed-41b6-a683-989b8a29be4f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:37.791140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0e900-1efd-48b6-9968-b8715ed5dfe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:37.791762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0e900-1efd-48b6-9968-b8715ed5dfe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.792062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0e900-1efd-48b6-9968-b8715ed5dfe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.792342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0e900-1efd-48b6-9968-b8715ed5dfe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:37.868736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3d5c6-a1ed-41b6-a683-989b8a29be4f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:37.872481 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0e900-1efd-48b6-9968-b8715ed5dfe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:37.874259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0e900-1efd-48b6-9968-b8715ed5dfe0 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:37.875448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992117d-a281-49c0-84e6-2874d5ce5a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:37.876214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992117d-a281-49c0-84e6-2874d5ce5a1b admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.876564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992117d-a281-49c0-84e6-2874d5ce5a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.876922 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992117d-a281-49c0-84e6-2874d5ce5a1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:37.877647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992117d-a281-49c0-84e6-2874d5ce5a1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:37.879985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992117d-a281-49c0-84e6-2874d5ce5a1b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:37.887672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1170/2316] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:37 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:37.964470 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:37.968168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0e900-1efd-48b6-9968-b8715ed5dfe0 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:37.979211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0e900-1efd-48b6-9968-b8715ed5dfe0 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:37.986030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f5f6f-c965-403a-9346-c8f34fbc9066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:37.986674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f5f6f-c965-403a-9346-c8f34fbc9066 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:37.986952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f5f6f-c965-403a-9346-c8f34fbc9066 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:37.987228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f5f6f-c965-403a-9346-c8f34fbc9066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:38.004367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1147/2317] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:37 2022] POST /identity/v3/projects => generated 387 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:38.010543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992117d-a281-49c0-84e6-2874d5ce5a1b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:38.060889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f5f6f-c965-403a-9346-c8f34fbc9066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:38.062384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f5f6f-c965-403a-9346-c8f34fbc9066 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:38.073095 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1171/2318] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:39:38.089682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992117d-a281-49c0-84e6-2874d5ce5a1b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:38.098075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6617c-ef88-46cc-b7ce-6d63ed1808d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:38.098075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6617c-ef88-46cc-b7ce-6d63ed1808d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:38.098075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6617c-ef88-46cc-b7ce-6d63ed1808d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:38.098075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6617c-ef88-46cc-b7ce-6d63ed1808d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:38.098402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6617c-ef88-46cc-b7ce-6d63ed1808d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:38.099252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6617c-ef88-46cc-b7ce-6d63ed1808d2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:38.196510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1148/2319] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:38 2022] POST /identity/v3/users => generated 300 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:38.197335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f5f6f-c965-403a-9346-c8f34fbc9066 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:38.216815 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f5f6f-c965-403a-9346-c8f34fbc9066 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:38.224828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0db87-011a-4f49-aeba-41f9d7fa8b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:38.225618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0db87-011a-4f49-aeba-41f9d7fa8b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:38.226205 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0db87-011a-4f49-aeba-41f9d7fa8b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:38.227207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0db87-011a-4f49-aeba-41f9d7fa8b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:38.228300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0db87-011a-4f49-aeba-41f9d7fa8b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:38.230179 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0db87-011a-4f49-aeba-41f9d7fa8b10 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:38.256236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c15f4b-2c2a-49cb-9e2d-92d0a332432b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:38.256700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c15f4b-2c2a-49cb-9e2d-92d0a332432b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:38.256780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c15f4b-2c2a-49cb-9e2d-92d0a332432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:38.269116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1172/2320] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:38.270444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0db87-011a-4f49-aeba-41f9d7fa8b10 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:38.293107 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0db87-011a-4f49-aeba-41f9d7fa8b10 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:38.300386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43119e90-7088-4a7c-9643-a46efaba8b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:38.301286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43119e90-7088-4a7c-9643-a46efaba8b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:38.301949 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43119e90-7088-4a7c-9643-a46efaba8b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:38.302410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43119e90-7088-4a7c-9643-a46efaba8b3a admin admin] PATH_INFO: `/v3/projects/6d724cdaa6ec40209b5571836cd72462/users/6e9baab35c8f40158bb0666b5f790c46/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:38.317010 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-88c15f4b-2c2a-49cb-9e2d-92d0a332432b None None] MFA Rules not processed for user `def1da13ff2a47ada8e5f453061e3f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:38.318257 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-88c15f4b-2c2a-49cb-9e2d-92d0a332432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:38.323334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:38.325450 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43119e90-7088-4a7c-9643-a46efaba8b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d724cdaa6ec40209b5571836cd72462, user_id=6e9baab35c8f40158bb0666b5f790c46, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:38.329660 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43119e90-7088-4a7c-9643-a46efaba8b3a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:38.407249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1173/2321] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:38 2022] PUT /identity/v3/projects/6d724cdaa6ec40209b5571836cd72462/users/6e9baab35c8f40158bb0666b5f790c46/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:38.408317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43119e90-7088-4a7c-9643-a46efaba8b3a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:38.490725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1149/2322] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:38.560207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43119e90-7088-4a7c-9643-a46efaba8b3a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:38.566783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bf8ef-3cde-439a-9126-9310f24527af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:38.567829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bf8ef-3cde-439a-9126-9310f24527af admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:38.568101 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bf8ef-3cde-439a-9126-9310f24527af admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:38.568360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bf8ef-3cde-439a-9126-9310f24527af admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:38.569010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bf8ef-3cde-439a-9126-9310f24527af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:38.571322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bf8ef-3cde-439a-9126-9310f24527af admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:38.577808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c15f4b-2c2a-49cb-9e2d-92d0a332432b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:38.579485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-88c15f4b-2c2a-49cb-9e2d-92d0a332432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:38.584185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1174/2323] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:38.588389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bf8ef-3cde-439a-9126-9310f24527af admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:38.601303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bf8ef-3cde-439a-9126-9310f24527af admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:38.610448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64798f4-2e75-4791-861b-5503cb98a59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:38.611127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64798f4-2e75-4791-861b-5503cb98a59d admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:38.611404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64798f4-2e75-4791-861b-5503cb98a59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:38.611677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64798f4-2e75-4791-861b-5503cb98a59d admin admin] PATH_INFO: `/v3/projects/6d724cdaa6ec40209b5571836cd72462/users/6e9baab35c8f40158bb0666b5f790c46/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:38.615226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64798f4-2e75-4791-861b-5503cb98a59d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d724cdaa6ec40209b5571836cd72462, user_id=6e9baab35c8f40158bb0666b5f790c46, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:38.622962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64798f4-2e75-4791-861b-5503cb98a59d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:38.640061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1175/2324] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:38 2022] PUT /identity/v3/projects/6d724cdaa6ec40209b5571836cd72462/users/6e9baab35c8f40158bb0666b5f790c46/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:38.649148 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc1ec-0904-4f3f-9a60-04c7b8e6f00c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:38.649435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc1ec-0904-4f3f-9a60-04c7b8e6f00c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:38.650157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc1ec-0904-4f3f-9a60-04c7b8e6f00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:38.670852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c15f4b-2c2a-49cb-9e2d-92d0a332432b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:38.686238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-51dcc1ec-0904-4f3f-9a60-04c7b8e6f00c None None] MFA Rules not processed for user `6e9baab35c8f40158bb0666b5f790c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:38.688577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-51dcc1ec-0904-4f3f-9a60-04c7b8e6f00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:38.736904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9d762-48ae-48e0-97bc-6e05d1ce4a5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:38.737513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9d762-48ae-48e0-97bc-6e05d1ce4a5c service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:38.737794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9d762-48ae-48e0-97bc-6e05d1ce4a5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:38.738067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9d762-48ae-48e0-97bc-6e05d1ce4a5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:38.767724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1176/2325] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:38.797429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de9d762-48ae-48e0-97bc-6e05d1ce4a5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:38.798934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de9d762-48ae-48e0-97bc-6e05d1ce4a5c service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:38.809358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:38.811829 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1150/2326] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:38.924652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcc1ec-0904-4f3f-9a60-04c7b8e6f00c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:38.936330 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcc1ec-0904-4f3f-9a60-04c7b8e6f00c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:38.947897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362411c9-6222-4ea0-b86c-d2e98cfda3ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:38.949320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362411c9-6222-4ea0-b86c-d2e98cfda3ca service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:38.949832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362411c9-6222-4ea0-b86c-d2e98cfda3ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:38.950552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362411c9-6222-4ea0-b86c-d2e98cfda3ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:38.953020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a25b02-5e55-4070-b61a-7e87828e5dc3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:38.953311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a25b02-5e55-4070-b61a-7e87828e5dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:38.955417 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a25b02-5e55-4070-b61a-7e87828e5dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:38.966058 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362411c9-6222-4ea0-b86c-d2e98cfda3ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:38.967820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362411c9-6222-4ea0-b86c-d2e98cfda3ca service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:38.978380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e6a25b02-5e55-4070-b61a-7e87828e5dc3 None None] MFA Rules not processed for user `966045fbe69f43b4b1a825da3ad571b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:38.979812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e6a25b02-5e55-4070-b61a-7e87828e5dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:38.984775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1177/2327] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:39:39.044542 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1151/2328] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:39.380858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:39.727427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362411c9-6222-4ea0-b86c-d2e98cfda3ca service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:39.852381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362411c9-6222-4ea0-b86c-d2e98cfda3ca service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:39.858118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:39.862057 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:39.862471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:39.863466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] PATH_INFO: `/v3/users/d321c9d029ef4e9fb67c2a09a24d82e8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:39.865857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d321c9d029ef4e9fb67c2a09a24d82e8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:39.868563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:39.905932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:40.062758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd321c9d029ef4e9fb67c2a09a24d82e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:40.198092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd321c9d029ef4e9fb67c2a09a24d82e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:40.212069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd321c9d029ef4e9fb67c2a09a24d82e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:40.219108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd321c9d029ef4e9fb67c2a09a24d82e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:40.219973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd321c9d029ef4e9fb67c2a09a24d82e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:40.228054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1178/2329] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:39 2022] DELETE /identity/v3/users/d321c9d029ef4e9fb67c2a09a24d82e8 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:40.232622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367803b-aefd-4cd6-8d5c-f6bffaba4da1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:40.233255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367803b-aefd-4cd6-8d5c-f6bffaba4da1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:40.233518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367803b-aefd-4cd6-8d5c-f6bffaba4da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:40.252848 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9367803b-aefd-4cd6-8d5c-f6bffaba4da1 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:40.254055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9367803b-aefd-4cd6-8d5c-f6bffaba4da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:40.319719 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1152/2330] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:40.330650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:40.332217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:40.397844 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56500cd-26b8-469e-acc1-88c2e68d53a4 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:40.403032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7120f-5663-4967-91bc-5e6c4686a011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:40.403698 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7120f-5663-4967-91bc-5e6c4686a011 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:40.403976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7120f-5663-4967-91bc-5e6c4686a011 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:40.404323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7120f-5663-4967-91bc-5e6c4686a011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:40.416254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7120f-5663-4967-91bc-5e6c4686a011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:40.417851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7120f-5663-4967-91bc-5e6c4686a011 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:40.430299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1179/2331] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:40.546805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:40.670291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98266a96-6dd9-48d1-9576-adccd46c1a5d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:40.670575 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98266a96-6dd9-48d1-9576-adccd46c1a5d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:40.670857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98266a96-6dd9-48d1-9576-adccd46c1a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:40.705978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-98266a96-6dd9-48d1-9576-adccd46c1a5d None None] MFA Rules not processed for user `def1da13ff2a47ada8e5f453061e3f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:40.707298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-98266a96-6dd9-48d1-9576-adccd46c1a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:40.812367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1153/2332] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:40.912419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:40.924911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7120f-5663-4967-91bc-5e6c4686a011 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:40.926442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2ca7120f-5663-4967-91bc-5e6c4686a011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:40.974892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7120f-5663-4967-91bc-5e6c4686a011 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:40.991557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:40.991557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:40.991557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:40.991557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] PATH_INFO: `/v3/projects/8f5876445c2541c8b0051da9eef4a090` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:40.991862 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f5876445c2541c8b0051da9eef4a090)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:40.993449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:41.178486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:41.322672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f5876445c2541c8b0051da9eef4a090', 'request_id': 'req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:41.329750 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f5876445c2541c8b0051da9eef4a090', 'request_id': 'req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:41.415805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f5876445c2541c8b0051da9eef4a090', 'request_id': 'req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:41.428415 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] The token cache is being invalidate because project 8f5876445c2541c8b0051da9eef4a090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:41.428834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:41.430284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:41.431694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1180/2333] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:40 2022] DELETE /identity/v3/projects/8f5876445c2541c8b0051da9eef4a090 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:41.441921 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98266a96-6dd9-48d1-9576-adccd46c1a5d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:41.461119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98266a96-6dd9-48d1-9576-adccd46c1a5d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:41.547503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e6145-7ac0-4274-82cc-8f7f3bb1a73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:41.548275 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e6145-7ac0-4274-82cc-8f7f3bb1a73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:41.548621 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e6145-7ac0-4274-82cc-8f7f3bb1a73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:41.548958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e6145-7ac0-4274-82cc-8f7f3bb1a73a admin admin] PATH_INFO: `/v3/users/6e9baab35c8f40158bb0666b5f790c46` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:41.550153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e6145-7ac0-4274-82cc-8f7f3bb1a73a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9baab35c8f40158bb0666b5f790c46)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:41.552159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e6145-7ac0-4274-82cc-8f7f3bb1a73a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:41.683698 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:41.772008 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-849e6145-7ac0-4274-82cc-8f7f3bb1a73a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9baab35c8f40158bb0666b5f790c46'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:41.843214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-849e6145-7ac0-4274-82cc-8f7f3bb1a73a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9baab35c8f40158bb0666b5f790c46'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:41.866910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-849e6145-7ac0-4274-82cc-8f7f3bb1a73a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9baab35c8f40158bb0666b5f790c46'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:41.866910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-849e6145-7ac0-4274-82cc-8f7f3bb1a73a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9baab35c8f40158bb0666b5f790c46'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:41.866910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-849e6145-7ac0-4274-82cc-8f7f3bb1a73a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9baab35c8f40158bb0666b5f790c46'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:41.879975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1154/2334] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:41 2022] DELETE /identity/v3/users/6e9baab35c8f40158bb0666b5f790c46 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:42.073160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:42.570303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:42.572524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:42.595314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e2cd8-6d84-4d14-8304-a9fe6920ca81 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:42.699786 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:42.700435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:42.700950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:42.701558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] PATH_INFO: `/v3/projects/6d724cdaa6ec40209b5571836cd72462` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:42.703675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d724cdaa6ec40209b5571836cd72462)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:42.706016 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:42.723323 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:42.864229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d724cdaa6ec40209b5571836cd72462', 'request_id': 'req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:42.871781 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d724cdaa6ec40209b5571836cd72462', 'request_id': 'req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:42.991227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d724cdaa6ec40209b5571836cd72462', 'request_id': 'req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:42.994304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:43.006089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] The token cache is being invalidate because project 6d724cdaa6ec40209b5571836cd72462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:43.006381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:43.007119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b701a23-8d7a-421b-a8f7-25e9a94c80a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:43.009222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1181/2335] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:42 2022] DELETE /identity/v3/projects/6d724cdaa6ec40209b5571836cd72462 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:43.014340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e23b6f-90b7-401f-902d-b544de9f4004 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:43.018845 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e23b6f-90b7-401f-902d-b544de9f4004 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:43.018845 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e23b6f-90b7-401f-902d-b544de9f4004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:43.035536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-55e23b6f-90b7-401f-902d-b544de9f4004 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:43.037071 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-55e23b6f-90b7-401f-902d-b544de9f4004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:43.127950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1155/2336] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:43.134085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa79dea-82af-4696-b482-e2872868246f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:43.134371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa79dea-82af-4696-b482-e2872868246f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:43.134649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa79dea-82af-4696-b482-e2872868246f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:43.162502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-daa79dea-82af-4696-b482-e2872868246f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:43.164569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-daa79dea-82af-4696-b482-e2872868246f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:43.178265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1182/2337] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:43.183580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e23b6f-90b7-401f-902d-b544de9f4004 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:43.197212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e23b6f-90b7-401f-902d-b544de9f4004 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:43.204254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28808e-ffc3-4f04-85cd-2ce1e424d287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:43.205242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28808e-ffc3-4f04-85cd-2ce1e424d287 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:43.205527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28808e-ffc3-4f04-85cd-2ce1e424d287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:43.205802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28808e-ffc3-4f04-85cd-2ce1e424d287 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:43.206508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28808e-ffc3-4f04-85cd-2ce1e424d287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:43.207454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28808e-ffc3-4f04-85cd-2ce1e424d287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:43.213199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28808e-ffc3-4f04-85cd-2ce1e424d287 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:43.228206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1156/2338] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:43.231103 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa79dea-82af-4696-b482-e2872868246f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:43.243716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa79dea-82af-4696-b482-e2872868246f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:43.252041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d3231-c2f2-421d-9c7f-b09b7888d4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:43.252680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d3231-c2f2-421d-9c7f-b09b7888d4dc admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:43.252957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d3231-c2f2-421d-9c7f-b09b7888d4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:43.253228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d3231-c2f2-421d-9c7f-b09b7888d4dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:43.253879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d3231-c2f2-421d-9c7f-b09b7888d4dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:43.255799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d3231-c2f2-421d-9c7f-b09b7888d4dc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:43.371390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1183/2339] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:43 2022] POST /identity/v3/projects => generated 389 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:43.375514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28808e-ffc3-4f04-85cd-2ce1e424d287 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:43.390401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28808e-ffc3-4f04-85cd-2ce1e424d287 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:43.460919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85504413-d297-4448-8ac6-793b5728b847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:43.461543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85504413-d297-4448-8ac6-793b5728b847 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:43.461825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85504413-d297-4448-8ac6-793b5728b847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:43.462166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85504413-d297-4448-8ac6-793b5728b847 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:43.462799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85504413-d297-4448-8ac6-793b5728b847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:43.466182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85504413-d297-4448-8ac6-793b5728b847 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:43.550688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1157/2340] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:43 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:43.555237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d3231-c2f2-421d-9c7f-b09b7888d4dc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:43.566496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d3231-c2f2-421d-9c7f-b09b7888d4dc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:43.574774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfcb1d-dce7-4b11-958c-4a2b5747efcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:43.576062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfcb1d-dce7-4b11-958c-4a2b5747efcd admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:43.576534 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfcb1d-dce7-4b11-958c-4a2b5747efcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:43.576959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfcb1d-dce7-4b11-958c-4a2b5747efcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:43.577800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfcb1d-dce7-4b11-958c-4a2b5747efcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:43.579689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfcb1d-dce7-4b11-958c-4a2b5747efcd admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:43.597055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1184/2341] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:43.597209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85504413-d297-4448-8ac6-793b5728b847 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:43.610971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85504413-d297-4448-8ac6-793b5728b847 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:43.611959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:43.617607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79fbd4-5056-48e1-a56b-f54fed77a5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:43.618706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79fbd4-5056-48e1-a56b-f54fed77a5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:43.618996 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79fbd4-5056-48e1-a56b-f54fed77a5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:43.619276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79fbd4-5056-48e1-a56b-f54fed77a5e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:43.619934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79fbd4-5056-48e1-a56b-f54fed77a5e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:43.621751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79fbd4-5056-48e1-a56b-f54fed77a5e3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:43.636065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1158/2342] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:43.638202 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfcb1d-dce7-4b11-958c-4a2b5747efcd admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:43.649037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfcb1d-dce7-4b11-958c-4a2b5747efcd admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:43.653658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a133472-13e3-4585-8dd5-501aa9fae29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:43.654527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a133472-13e3-4585-8dd5-501aa9fae29f admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:43.654802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a133472-13e3-4585-8dd5-501aa9fae29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:43.655075 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a133472-13e3-4585-8dd5-501aa9fae29f admin admin] PATH_INFO: `/v3/projects/fa9cc9f634d34119868bda2ce11378af/users/8b713f6d89cd46b9a1d8d932ea6288aa/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:43.671354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a133472-13e3-4585-8dd5-501aa9fae29f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa9cc9f634d34119868bda2ce11378af, user_id=8b713f6d89cd46b9a1d8d932ea6288aa, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:43.674085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a133472-13e3-4585-8dd5-501aa9fae29f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:43.724981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79fbd4-5056-48e1-a56b-f54fed77a5e3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:43.727424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6e79fbd4-5056-48e1-a56b-f54fed77a5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:43.741214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79fbd4-5056-48e1-a56b-f54fed77a5e3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:43.746187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1185/2343] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:43 2022] PUT /identity/v3/projects/fa9cc9f634d34119868bda2ce11378af/users/8b713f6d89cd46b9a1d8d932ea6288aa/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:43.758464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9083879-8fb8-40a8-9235-ab97855d7ea9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:43.759111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9083879-8fb8-40a8-9235-ab97855d7ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:43.759406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9083879-8fb8-40a8-9235-ab97855d7ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:43.794557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e9083879-8fb8-40a8-9235-ab97855d7ea9 None None] MFA Rules not processed for user `8b713f6d89cd46b9a1d8d932ea6288aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:43.796126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e9083879-8fb8-40a8-9235-ab97855d7ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:43.827300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:43.828218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:43.834287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:43.834287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] PATH_INFO: `/v3/users/8faa3c70c6a543b58d71980e8f418310` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:43.834287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8faa3c70c6a543b58d71980e8f418310)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:43.834287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:43.897818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1186/2344] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:43.908953 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9083879-8fb8-40a8-9235-ab97855d7ea9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:43.910416 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e9083879-8fb8-40a8-9235-ab97855d7ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:43.978156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9083879-8fb8-40a8-9235-ab97855d7ea9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:43.981988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcbb36-3fcb-428e-a439-a227ea0d315f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:43.982606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcbb36-3fcb-428e-a439-a227ea0d315f service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:43.982883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcbb36-3fcb-428e-a439-a227ea0d315f service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:43.983204 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcbb36-3fcb-428e-a439-a227ea0d315f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:43.993691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdcbb36-3fcb-428e-a439-a227ea0d315f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:43.995138 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdcbb36-3fcb-428e-a439-a227ea0d315f service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:44.004883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1187/2345] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:44.120083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:44.140608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8faa3c70c6a543b58d71980e8f418310'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:44.265702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8faa3c70c6a543b58d71980e8f418310'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:44.281876 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8faa3c70c6a543b58d71980e8f418310'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:44.284398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8faa3c70c6a543b58d71980e8f418310'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:44.285094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8faa3c70c6a543b58d71980e8f418310'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:44.302167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1159/2346] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:43 2022] DELETE /identity/v3/users/8faa3c70c6a543b58d71980e8f418310 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:44.306766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32f8d4-6cfc-40cf-9402-d8457ea1bb84 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:44.307167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32f8d4-6cfc-40cf-9402-d8457ea1bb84 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:44.307493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32f8d4-6cfc-40cf-9402-d8457ea1bb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:44.327292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7f32f8d4-6cfc-40cf-9402-d8457ea1bb84 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:44.328546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7f32f8d4-6cfc-40cf-9402-d8457ea1bb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:44.370076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:44.453574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1188/2347] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:44.465580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:44.467144 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:44.600725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fccf0-7890-4eec-9ee4-00f9576ac5e1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:44.607260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e006ac8-0f99-4e2b-8c09-91b2fd2fc3cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:44.608039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e006ac8-0f99-4e2b-8c09-91b2fd2fc3cc service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:44.608601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e006ac8-0f99-4e2b-8c09-91b2fd2fc3cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:44.614276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e006ac8-0f99-4e2b-8c09-91b2fd2fc3cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:44.619197 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e006ac8-0f99-4e2b-8c09-91b2fd2fc3cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:44.620760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e006ac8-0f99-4e2b-8c09-91b2fd2fc3cc service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:44.658550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1160/2348] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:45.248261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32f8d4-6cfc-40cf-9402-d8457ea1bb84 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:45.248261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7f32f8d4-6cfc-40cf-9402-d8457ea1bb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:45.282008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32f8d4-6cfc-40cf-9402-d8457ea1bb84 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:45.301865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e75ae-fce6-442c-8702-2eb87cb26f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:45.302539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e75ae-fce6-442c-8702-2eb87cb26f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:45.302813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e75ae-fce6-442c-8702-2eb87cb26f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:45.303131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e75ae-fce6-442c-8702-2eb87cb26f5c admin admin] PATH_INFO: `/v3/users/9c90c556d4c04951819794cf1cc11493` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:45.305187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e75ae-fce6-442c-8702-2eb87cb26f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c90c556d4c04951819794cf1cc11493)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:45.309622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e75ae-fce6-442c-8702-2eb87cb26f5c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:45.310797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e006ac8-0f99-4e2b-8c09-91b2fd2fc3cc service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:45.312384 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4e006ac8-0f99-4e2b-8c09-91b2fd2fc3cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:45.331826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e006ac8-0f99-4e2b-8c09-91b2fd2fc3cc service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:45.338241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:45.339612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:45.340254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:45.340749 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] PATH_INFO: `/v3/projects/d0b7aca06bfa4aabaf02b861161d04be` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:45.342507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0b7aca06bfa4aabaf02b861161d04be)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:45.345554 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:45.389653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:45.493155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-786e75ae-fce6-442c-8702-2eb87cb26f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c90c556d4c04951819794cf1cc11493'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:45.493594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0b7aca06bfa4aabaf02b861161d04be', 'request_id': 'req-7df76757-2097-4478-b86a-9c4625dd9760'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:45.501245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0b7aca06bfa4aabaf02b861161d04be', 'request_id': 'req-7df76757-2097-4478-b86a-9c4625dd9760'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:45.557424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-786e75ae-fce6-442c-8702-2eb87cb26f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c90c556d4c04951819794cf1cc11493'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:45.571782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-786e75ae-fce6-442c-8702-2eb87cb26f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c90c556d4c04951819794cf1cc11493'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:45.578771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-786e75ae-fce6-442c-8702-2eb87cb26f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c90c556d4c04951819794cf1cc11493'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:45.579502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-786e75ae-fce6-442c-8702-2eb87cb26f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c90c556d4c04951819794cf1cc11493'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:45.588605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1189/2349] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:45 2022] DELETE /identity/v3/users/9c90c556d4c04951819794cf1cc11493 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:45.590940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7ed18-a64e-4e47-9088-f7d46c75a16e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:45.592281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7ed18-a64e-4e47-9088-f7d46c75a16e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:45.592564 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7ed18-a64e-4e47-9088-f7d46c75a16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:45.612043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-21f7ed18-a64e-4e47-9088-f7d46c75a16e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:45.613889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-21f7ed18-a64e-4e47-9088-f7d46c75a16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:45.637927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0b7aca06bfa4aabaf02b861161d04be', 'request_id': 'req-7df76757-2097-4478-b86a-9c4625dd9760'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:45.651010 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] The token cache is being invalidate because project d0b7aca06bfa4aabaf02b861161d04be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:45.651304 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:45.651955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:45.654262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1161/2350] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:45 2022] DELETE /identity/v3/projects/d0b7aca06bfa4aabaf02b861161d04be => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:45.657427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:45.658910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:45.676546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1190/2351] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:45.691281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7ed18-a64e-4e47-9088-f7d46c75a16e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:45.692799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-21f7ed18-a64e-4e47-9088-f7d46c75a16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:45.731706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df76757-2097-4478-b86a-9c4625dd9760 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:45.737035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:45.737758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:45.738152 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:45.738495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] PATH_INFO: `/v3/users/ec238b08150e4846821d5a1a8ae9addc` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:45.739840 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec238b08150e4846821d5a1a8ae9addc)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:45.742021 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:45.760299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7ed18-a64e-4e47-9088-f7d46c75a16e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:45.765854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94e9e0-b281-4153-9b1e-eea452ad00af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:45.766589 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94e9e0-b281-4153-9b1e-eea452ad00af service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:45.767019 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94e9e0-b281-4153-9b1e-eea452ad00af service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:45.767392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94e9e0-b281-4153-9b1e-eea452ad00af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:45.779742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94e9e0-b281-4153-9b1e-eea452ad00af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:45.781826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94e9e0-b281-4153-9b1e-eea452ad00af service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:45.794090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1191/2352] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:45.839646 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:45.926040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec238b08150e4846821d5a1a8ae9addc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:45.998261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec238b08150e4846821d5a1a8ae9addc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.014414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec238b08150e4846821d5a1a8ae9addc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.015347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec238b08150e4846821d5a1a8ae9addc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.015988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec238b08150e4846821d5a1a8ae9addc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.032179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1162/2353] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:45 2022] DELETE /identity/v3/users/ec238b08150e4846821d5a1a8ae9addc => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:46.236244 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94e9e0-b281-4153-9b1e-eea452ad00af service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:46.242808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0b94e9e0-b281-4153-9b1e-eea452ad00af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:46.272231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94e9e0-b281-4153-9b1e-eea452ad00af service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:46.357349 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:46.358115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:46.358833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:46.358833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] PATH_INFO: `/v3/projects/d983573bf6424422a28e763405a98c7e` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:46.360185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d983573bf6424422a28e763405a98c7e)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:46.364282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:46.490530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:46.493240 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:46.524561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f88d8-e45a-4fcb-bf4e-a05db4d2b36a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:46.543215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd983573bf6424422a28e763405a98c7e', 'request_id': 'req-6e3f94ef-a258-45cc-a589-28cf9693058c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.576383 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd983573bf6424422a28e763405a98c7e', 'request_id': 'req-6e3f94ef-a258-45cc-a589-28cf9693058c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.609128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:46.610068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:46.610563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:46.611070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] PATH_INFO: `/v3/projects/ad763b4b13e246fe8bb4f3eddd016b2a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:46.612489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad763b4b13e246fe8bb4f3eddd016b2a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:46.614638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:46.683290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd983573bf6424422a28e763405a98c7e', 'request_id': 'req-6e3f94ef-a258-45cc-a589-28cf9693058c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.696811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] The token cache is being invalidate because project d983573bf6424422a28e763405a98c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:46.697468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.698408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6e3f94ef-a258-45cc-a589-28cf9693058c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.700064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1192/2354] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:46 2022] DELETE /identity/v3/projects/d983573bf6424422a28e763405a98c7e => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:46.704925 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092debf5-79ae-4661-9ebb-a504c1e07a8b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:46.705314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092debf5-79ae-4661-9ebb-a504c1e07a8b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:46.705857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092debf5-79ae-4661-9ebb-a504c1e07a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:46.731151 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-092debf5-79ae-4661-9ebb-a504c1e07a8b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:46.737240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-092debf5-79ae-4661-9ebb-a504c1e07a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:46.740402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad763b4b13e246fe8bb4f3eddd016b2a', 'request_id': 'req-abaf031f-621f-4472-a7b1-2ca9ad016b60'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.749643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad763b4b13e246fe8bb4f3eddd016b2a', 'request_id': 'req-abaf031f-621f-4472-a7b1-2ca9ad016b60'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.785894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:46.801181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1193/2355] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:46.809403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2567ff4-f013-4c1c-98f3-cfdd70f376cc None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:46.809692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2567ff4-f013-4c1c-98f3-cfdd70f376cc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:46.809968 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2567ff4-f013-4c1c-98f3-cfdd70f376cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:46.830557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c2567ff4-f013-4c1c-98f3-cfdd70f376cc None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:46.831503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad763b4b13e246fe8bb4f3eddd016b2a', 'request_id': 'req-abaf031f-621f-4472-a7b1-2ca9ad016b60'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.831804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c2567ff4-f013-4c1c-98f3-cfdd70f376cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:46.844530 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] The token cache is being invalidate because project ad763b4b13e246fe8bb4f3eddd016b2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:46.844755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1194/2356] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:46.844846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.845451 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-abaf031f-621f-4472-a7b1-2ca9ad016b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:46.845561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2567ff4-f013-4c1c-98f3-cfdd70f376cc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:46.851911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1163/2357] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:46 2022] DELETE /identity/v3/projects/ad763b4b13e246fe8bb4f3eddd016b2a => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:46.853739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2ef16-446f-4cb4-b78c-1aae829915c5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:46.854185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2ef16-446f-4cb4-b78c-1aae829915c5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:46.854463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2ef16-446f-4cb4-b78c-1aae829915c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:46.856238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2567ff4-f013-4c1c-98f3-cfdd70f376cc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:46.861500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2647ab-e9ee-4629-950e-ff3382ddcb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:46.862267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2647ab-e9ee-4629-950e-ff3382ddcb9f admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:46.862613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2647ab-e9ee-4629-950e-ff3382ddcb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:46.874183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2647ab-e9ee-4629-950e-ff3382ddcb9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:46.874183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2647ab-e9ee-4629-950e-ff3382ddcb9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:46.874183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2647ab-e9ee-4629-950e-ff3382ddcb9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:46.874183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2647ab-e9ee-4629-950e-ff3382ddcb9f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:46.880673 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9ae2ef16-446f-4cb4-b78c-1aae829915c5 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:46.882108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9ae2ef16-446f-4cb4-b78c-1aae829915c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:46.882532 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1195/2358] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:46.883472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2647ab-e9ee-4629-950e-ff3382ddcb9f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:46.885462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bb2647ab-e9ee-4629-950e-ff3382ddcb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:46.896540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1164/2359] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:46.906446 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2647ab-e9ee-4629-950e-ff3382ddcb9f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:46.908128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec6f3a-2388-4768-be28-478f354e6ef0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:46.908515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec6f3a-2388-4768-be28-478f354e6ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:46.908940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec6f3a-2388-4768-be28-478f354e6ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:46.910902 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665582c7-7d20-4fd2-9f31-50d0571acc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:46.911839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665582c7-7d20-4fd2-9f31-50d0571acc90 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:46.912152 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665582c7-7d20-4fd2-9f31-50d0571acc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:46.912432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665582c7-7d20-4fd2-9f31-50d0571acc90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:46.914316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665582c7-7d20-4fd2-9f31-50d0571acc90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:46.920847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665582c7-7d20-4fd2-9f31-50d0571acc90 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:46.938655 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-16ec6f3a-2388-4768-be28-478f354e6ef0 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:46.941604 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-16ec6f3a-2388-4768-be28-478f354e6ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:46.959325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1165/2360] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:46.966453 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec6f3a-2388-4768-be28-478f354e6ef0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:46.982219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec6f3a-2388-4768-be28-478f354e6ef0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:46.990414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19082a9-39c9-48f2-bd33-36f145f49b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:46.991040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19082a9-39c9-48f2-bd33-36f145f49b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:46.991320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19082a9-39c9-48f2-bd33-36f145f49b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:46.991600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19082a9-39c9-48f2-bd33-36f145f49b76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:46.992426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19082a9-39c9-48f2-bd33-36f145f49b76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:46.992741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19082a9-39c9-48f2-bd33-36f145f49b76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:46.994545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19082a9-39c9-48f2-bd33-36f145f49b76 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:47.007959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1166/2361] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:39:47.014794 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19082a9-39c9-48f2-bd33-36f145f49b76 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:47.027469 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19082a9-39c9-48f2-bd33-36f145f49b76 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:47.032060 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1196/2362] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:46 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:47.035276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665582c7-7d20-4fd2-9f31-50d0571acc90 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:47.078632 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:47.110702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c104aa-75f0-4453-9376-16d7b7e73b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:47.111333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c104aa-75f0-4453-9376-16d7b7e73b15 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.112465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c104aa-75f0-4453-9376-16d7b7e73b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.113766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c104aa-75f0-4453-9376-16d7b7e73b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:47.114464 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c104aa-75f0-4453-9376-16d7b7e73b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:47.119257 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c104aa-75f0-4453-9376-16d7b7e73b15 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:47.143998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665582c7-7d20-4fd2-9f31-50d0571acc90 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:47.158766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3066b1c-e434-4631-8e7b-72843cae3719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:47.159652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3066b1c-e434-4631-8e7b-72843cae3719 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.159754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3066b1c-e434-4631-8e7b-72843cae3719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.160041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3066b1c-e434-4631-8e7b-72843cae3719 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:47.160727 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3066b1c-e434-4631-8e7b-72843cae3719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:47.162810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3066b1c-e434-4631-8e7b-72843cae3719 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:47.247797 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1167/2363] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:47 2022] POST /identity/v3/projects => generated 359 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:47.252032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c104aa-75f0-4453-9376-16d7b7e73b15 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:47.272435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1197/2364] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:47 2022] POST /identity/v3/users => generated 290 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:47.287281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3066b1c-e434-4631-8e7b-72843cae3719 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:47.312603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3066b1c-e434-4631-8e7b-72843cae3719 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:47.334061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c104aa-75f0-4453-9376-16d7b7e73b15 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:47.340241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6057b-e476-4f44-b56d-9c1878044299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:47.340943 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6057b-e476-4f44-b56d-9c1878044299 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.341291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6057b-e476-4f44-b56d-9c1878044299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.341630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6057b-e476-4f44-b56d-9c1878044299 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:47.342513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6057b-e476-4f44-b56d-9c1878044299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:47.344345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6057b-e476-4f44-b56d-9c1878044299 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:47.374559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38c28f-013c-4485-b579-30debe169f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:47.375129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38c28f-013c-4485-b579-30debe169f4c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.375408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38c28f-013c-4485-b579-30debe169f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.375676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38c28f-013c-4485-b579-30debe169f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:47.376381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38c28f-013c-4485-b579-30debe169f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:47.377988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38c28f-013c-4485-b579-30debe169f4c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:47.388363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1198/2365] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:47.392344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38c28f-013c-4485-b579-30debe169f4c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:47.402305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38c28f-013c-4485-b579-30debe169f4c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:47.406909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a16ad-80f5-4ed1-b654-d01be2f94045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:47.407471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a16ad-80f5-4ed1-b654-d01be2f94045 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.407744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a16ad-80f5-4ed1-b654-d01be2f94045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.408039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a16ad-80f5-4ed1-b654-d01be2f94045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:47.408648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a16ad-80f5-4ed1-b654-d01be2f94045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:47.410177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a16ad-80f5-4ed1-b654-d01be2f94045 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:47.420796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1199/2366] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:47.424226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a16ad-80f5-4ed1-b654-d01be2f94045 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:47.433846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a16ad-80f5-4ed1-b654-d01be2f94045 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:47.447079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696359bd-6488-47c8-96d0-609eb650adfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:47.448151 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1168/2367] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:47 2022] POST /identity/v3/users => generated 280 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:47.449237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6057b-e476-4f44-b56d-9c1878044299 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:47.453548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696359bd-6488-47c8-96d0-609eb650adfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.453891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696359bd-6488-47c8-96d0-609eb650adfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.454178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696359bd-6488-47c8-96d0-609eb650adfd admin admin] PATH_INFO: `/v3/projects/6b3cfd456d8748b3a1db990d638340af/users/394c67bcc3864bb2aca2f4faec834029/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:47.463400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6057b-e476-4f44-b56d-9c1878044299 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:47.470286 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08473ea-cbc8-437f-be3d-a520ae49820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:47.471168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08473ea-cbc8-437f-be3d-a520ae49820a admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.471715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08473ea-cbc8-437f-be3d-a520ae49820a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.472256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08473ea-cbc8-437f-be3d-a520ae49820a admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:47.473117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08473ea-cbc8-437f-be3d-a520ae49820a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:47.475125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08473ea-cbc8-437f-be3d-a520ae49820a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:47.490335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1169/2368] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:47.492486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08473ea-cbc8-437f-be3d-a520ae49820a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:47.499304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696359bd-6488-47c8-96d0-609eb650adfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3cfd456d8748b3a1db990d638340af, user_id=394c67bcc3864bb2aca2f4faec834029, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:47.503879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696359bd-6488-47c8-96d0-609eb650adfd admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:47.517403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08473ea-cbc8-437f-be3d-a520ae49820a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:47.524980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb3fd0-aa6e-4bc0-aab5-d097225a2408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:47.525823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3fd0-aa6e-4bc0-aab5-d097225a2408 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.526103 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3fd0-aa6e-4bc0-aab5-d097225a2408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.526374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3fd0-aa6e-4bc0-aab5-d097225a2408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:47.527050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb3fd0-aa6e-4bc0-aab5-d097225a2408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:47.528790 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb3fd0-aa6e-4bc0-aab5-d097225a2408 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:47.540861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1170/2369] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:47.545508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb3fd0-aa6e-4bc0-aab5-d097225a2408 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:47.556761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb3fd0-aa6e-4bc0-aab5-d097225a2408 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:47.568237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205be08-1910-4ed1-901c-7282e051f1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:47.568237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205be08-1910-4ed1-901c-7282e051f1fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.568237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205be08-1910-4ed1-901c-7282e051f1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.568237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205be08-1910-4ed1-901c-7282e051f1fd admin admin] PATH_INFO: `/v3/projects/2cf2a0ff95da4697925951767ebdfd58/users/732732c601fb43208fd6db66b10104b7/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:47.581898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205be08-1910-4ed1-901c-7282e051f1fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cf2a0ff95da4697925951767ebdfd58, user_id=732732c601fb43208fd6db66b10104b7, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:47.587482 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205be08-1910-4ed1-901c-7282e051f1fd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:47.589296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1200/2370] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:47 2022] PUT /identity/v3/projects/6b3cfd456d8748b3a1db990d638340af/users/394c67bcc3864bb2aca2f4faec834029/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:47.600962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f52afe-8a54-47bb-b7d0-ec416726be4e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.600962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f52afe-8a54-47bb-b7d0-ec416726be4e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.600962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f52afe-8a54-47bb-b7d0-ec416726be4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:47.622718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-41f52afe-8a54-47bb-b7d0-ec416726be4e None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:47.624142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-41f52afe-8a54-47bb-b7d0-ec416726be4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:47.672523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1171/2371] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:47 2022] PUT /identity/v3/projects/2cf2a0ff95da4697925951767ebdfd58/users/732732c601fb43208fd6db66b10104b7/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:47.682991 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bad31a-42c0-45be-8edb-34db329fe571 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.683346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bad31a-42c0-45be-8edb-34db329fe571 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.683687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bad31a-42c0-45be-8edb-34db329fe571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:47.733977 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1201/2372] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:47.742768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-66bad31a-42c0-45be-8edb-34db329fe571 None None] MFA Rules not processed for user `732732c601fb43208fd6db66b10104b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:47.744667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-66bad31a-42c0-45be-8edb-34db329fe571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:47.818067 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f52afe-8a54-47bb-b7d0-ec416726be4e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:47.825857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-41f52afe-8a54-47bb-b7d0-ec416726be4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:47.873291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1172/2373] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:47.906112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bad31a-42c0-45be-8edb-34db329fe571 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:47.907569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-66bad31a-42c0-45be-8edb-34db329fe571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:47.918208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f52afe-8a54-47bb-b7d0-ec416726be4e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:47.974252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:47.993150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0d5f1-4937-4b43-8c0d-8871fe866758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:47.995080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d5f1-4937-4b43-8c0d-8871fe866758 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:47.995493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d5f1-4937-4b43-8c0d-8871fe866758 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:47.995815 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d5f1-4937-4b43-8c0d-8871fe866758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:48.020453 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bad31a-42c0-45be-8edb-34db329fe571 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:48.025056 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1485109-7cf4-4124-90a1-ee9d22f0300e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:48.025673 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1485109-7cf4-4124-90a1-ee9d22f0300e service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:48.025986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1485109-7cf4-4124-90a1-ee9d22f0300e service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:48.026815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1485109-7cf4-4124-90a1-ee9d22f0300e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:48.036546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1485109-7cf4-4124-90a1-ee9d22f0300e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:48.038119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1485109-7cf4-4124-90a1-ee9d22f0300e service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:48.049047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1173/2374] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:39:48.065450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1485109-7cf4-4124-90a1-ee9d22f0300e service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:48.074109 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0d5f1-4937-4b43-8c0d-8871fe866758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:48.075591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0d5f1-4937-4b43-8c0d-8871fe866758 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:48.077434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1485109-7cf4-4124-90a1-ee9d22f0300e service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:48.083254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:48.084051 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:48.089308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:48.089308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] PATH_INFO: `/v3/users/732732c601fb43208fd6db66b10104b7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:48.089308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=732732c601fb43208fd6db66b10104b7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:48.089598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1202/2375] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:48.089694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:48.232130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:48.302393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '732732c601fb43208fd6db66b10104b7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:48.391315 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '732732c601fb43208fd6db66b10104b7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:48.408627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '732732c601fb43208fd6db66b10104b7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:48.409654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '732732c601fb43208fd6db66b10104b7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:48.410389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '732732c601fb43208fd6db66b10104b7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:48.430297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1174/2376] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:48 2022] DELETE /identity/v3/users/732732c601fb43208fd6db66b10104b7 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:48.435887 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff7594-73b7-481c-a148-f4e3d1cdc3be None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:48.436446 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff7594-73b7-481c-a148-f4e3d1cdc3be None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:48.436722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff7594-73b7-481c-a148-f4e3d1cdc3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:48.459582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-43ff7594-73b7-481c-a148-f4e3d1cdc3be None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:48.461013 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-43ff7594-73b7-481c-a148-f4e3d1cdc3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:48.556940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1203/2377] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:48.573262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:48.575278 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:48.636925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25070f11-8234-4791-bab2-24de4e6a36a2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:48.640525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb800b0-813c-487f-85e3-034e207e4b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:48.641138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb800b0-813c-487f-85e3-034e207e4b12 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:48.641413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb800b0-813c-487f-85e3-034e207e4b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:48.641684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb800b0-813c-487f-85e3-034e207e4b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:48.650909 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb800b0-813c-487f-85e3-034e207e4b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:48.652367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb800b0-813c-487f-85e3-034e207e4b12 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:48.661974 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1175/2378] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:48.860716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff7594-73b7-481c-a148-f4e3d1cdc3be None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:48.862154 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-43ff7594-73b7-481c-a148-f4e3d1cdc3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:48.877296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff7594-73b7-481c-a148-f4e3d1cdc3be None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:48.882082 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:48.882766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:48.883045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:48.883356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] PATH_INFO: `/v3/users/f240242958e34cbf81846b4e0cf4f437` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:48.884798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f240242958e34cbf81846b4e0cf4f437)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:48.886735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:48.983561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:49.083040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:49.086726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f240242958e34cbf81846b4e0cf4f437'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.141516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f240242958e34cbf81846b4e0cf4f437'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.160210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f240242958e34cbf81846b4e0cf4f437'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.173967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f240242958e34cbf81846b4e0cf4f437'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.173967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f240242958e34cbf81846b4e0cf4f437'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.178111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1204/2379] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:48 2022] DELETE /identity/v3/users/f240242958e34cbf81846b4e0cf4f437 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:49.256673 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb800b0-813c-487f-85e3-034e207e4b12 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:49.259937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8eb800b0-813c-487f-85e3-034e207e4b12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:49.279457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb800b0-813c-487f-85e3-034e207e4b12 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:49.376206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:49.377086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:49.377382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:49.378674 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] PATH_INFO: `/v3/projects/2cf2a0ff95da4697925951767ebdfd58` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:49.381341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cf2a0ff95da4697925951767ebdfd58)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:49.385425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:49.510559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:49.513027 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:49.546682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b9bd8-7a5c-4a16-9b2c-f15074a2b1c7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:49.553714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cf2a0ff95da4697925951767ebdfd58', 'request_id': 'req-f3aac7f6-6830-4286-84cf-fd0dec5919a8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.569117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cf2a0ff95da4697925951767ebdfd58', 'request_id': 'req-f3aac7f6-6830-4286-84cf-fd0dec5919a8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.639773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:49.640461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:49.641430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:49.641959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] PATH_INFO: `/v3/projects/764aad91598f4b13a6967b2d1fb7e59d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:49.643266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] RBAC: Authorizing `identity:delete_project(project_id=764aad91598f4b13a6967b2d1fb7e59d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:49.645562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:49.678252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cf2a0ff95da4697925951767ebdfd58', 'request_id': 'req-f3aac7f6-6830-4286-84cf-fd0dec5919a8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.692641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] The token cache is being invalidate because project 2cf2a0ff95da4697925951767ebdfd58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:49.693261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.694145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f3aac7f6-6830-4286-84cf-fd0dec5919a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.696772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1176/2380] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:49 2022] DELETE /identity/v3/projects/2cf2a0ff95da4697925951767ebdfd58 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:49.702318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bca4d4-9d41-4720-a120-a1c63e6c47d1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:49.703021 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bca4d4-9d41-4720-a120-a1c63e6c47d1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:49.703447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bca4d4-9d41-4720-a120-a1c63e6c47d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:49.742215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-58bca4d4-9d41-4720-a120-a1c63e6c47d1 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:49.743597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-58bca4d4-9d41-4720-a120-a1c63e6c47d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:49.749445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:49.768285 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764aad91598f4b13a6967b2d1fb7e59d', 'request_id': 'req-99f6ce05-c621-4c67-91e6-fb881088828d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.779155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764aad91598f4b13a6967b2d1fb7e59d', 'request_id': 'req-99f6ce05-c621-4c67-91e6-fb881088828d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.840330 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1177/2381] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:49.850016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64973378-94b5-48bc-85e3-4e37dffcd264 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:49.850506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64973378-94b5-48bc-85e3-4e37dffcd264 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:49.850937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64973378-94b5-48bc-85e3-4e37dffcd264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:49.875235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-64973378-94b5-48bc-85e3-4e37dffcd264 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:49.876848 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-64973378-94b5-48bc-85e3-4e37dffcd264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:49.887776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764aad91598f4b13a6967b2d1fb7e59d', 'request_id': 'req-99f6ce05-c621-4c67-91e6-fb881088828d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.901365 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1178/2382] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:49.905657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64973378-94b5-48bc-85e3-4e37dffcd264 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:49.910924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] The token cache is being invalidate because project 764aad91598f4b13a6967b2d1fb7e59d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:49.912279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.914142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:49.922460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1205/2383] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:49 2022] DELETE /identity/v3/projects/764aad91598f4b13a6967b2d1fb7e59d => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:49.926273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64973378-94b5-48bc-85e3-4e37dffcd264 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:49.926776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:49.928859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:49.933102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e784b-533a-4bd6-a0a8-7e74e6bafb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:49.933954 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e784b-533a-4bd6-a0a8-7e74e6bafb2f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:49.934418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e784b-533a-4bd6-a0a8-7e74e6bafb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:49.934877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e784b-533a-4bd6-a0a8-7e74e6bafb2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:49.936049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e784b-533a-4bd6-a0a8-7e74e6bafb2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:49.936516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e784b-533a-4bd6-a0a8-7e74e6bafb2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:49.938737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e784b-533a-4bd6-a0a8-7e74e6bafb2f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:49.946339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6ce05-c621-4c67-91e6-fb881088828d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:49.952005 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:49.952905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:49.953407 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:49.953907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] PATH_INFO: `/v3/users/50b3d596dc7345108154b9a4e7b6d4e8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:49.955213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b3d596dc7345108154b9a4e7b6d4e8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:49.958078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:49.959817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1179/2384] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:49.963037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8b005-2be5-4ae0-9dab-628c6dfe598a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:49.963520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8b005-2be5-4ae0-9dab-628c6dfe598a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:49.963958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8b005-2be5-4ae0-9dab-628c6dfe598a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:49.987706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6dd8b005-2be5-4ae0-9dab-628c6dfe598a None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:49.989287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6dd8b005-2be5-4ae0-9dab-628c6dfe598a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:50.092505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1180/2385] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:39:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:50.093268 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8b005-2be5-4ae0-9dab-628c6dfe598a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:50.095185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6dd8b005-2be5-4ae0-9dab-628c6dfe598a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:50.115328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8b005-2be5-4ae0-9dab-628c6dfe598a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:50.121320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e14381-0e4d-4538-8469-5942d12fcecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:50.122879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e14381-0e4d-4538-8469-5942d12fcecc admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:50.122879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e14381-0e4d-4538-8469-5942d12fcecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:50.122879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e14381-0e4d-4538-8469-5942d12fcecc admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:50.123584 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e14381-0e4d-4538-8469-5942d12fcecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:50.125750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e14381-0e4d-4538-8469-5942d12fcecc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:50.156157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b3d596dc7345108154b9a4e7b6d4e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:50.240461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b3d596dc7345108154b9a4e7b6d4e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:50.252822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1181/2386] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:50 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:50.252822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e14381-0e4d-4538-8469-5942d12fcecc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:50.259779 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-54e14381-0e4d-4538-8469-5942d12fcecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:50.262795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b3d596dc7345108154b9a4e7b6d4e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:50.266773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:50.285341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b3d596dc7345108154b9a4e7b6d4e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:50.286118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b3d596dc7345108154b9a4e7b6d4e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:50.304943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1206/2387] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:39:49 2022] DELETE /identity/v3/users/50b3d596dc7345108154b9a4e7b6d4e8 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:50.306981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:50.309807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:50.388579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a2528-ed65-4e8b-82b8-aaedf326c600 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:50.394725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23ac18-339c-48b2-8b7a-04f8f99afaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:50.394725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23ac18-339c-48b2-8b7a-04f8f99afaf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:50.394725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23ac18-339c-48b2-8b7a-04f8f99afaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:50.395117 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23ac18-339c-48b2-8b7a-04f8f99afaf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:50.395641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23ac18-339c-48b2-8b7a-04f8f99afaf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:50.397695 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23ac18-339c-48b2-8b7a-04f8f99afaf5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:50.424620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e14381-0e4d-4538-8469-5942d12fcecc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:50.507930 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1207/2388] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:50 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:50.513183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23ac18-339c-48b2-8b7a-04f8f99afaf5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:50.528488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23ac18-339c-48b2-8b7a-04f8f99afaf5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:50.553284 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c0d6e-a4dc-418a-a018-ede5a2ac0dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:50.554456 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c0d6e-a4dc-418a-a018-ede5a2ac0dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:50.554894 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c0d6e-a4dc-418a-a018-ede5a2ac0dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:50.555320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c0d6e-a4dc-418a-a018-ede5a2ac0dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:50.556429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c0d6e-a4dc-418a-a018-ede5a2ac0dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:50.558611 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c0d6e-a4dc-418a-a018-ede5a2ac0dd0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:50.573735 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9db2b-7c57-4ac0-bf7a-c58e80d6bba5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:50.574366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9db2b-7c57-4ac0-bf7a-c58e80d6bba5 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:50.574642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9db2b-7c57-4ac0-bf7a-c58e80d6bba5 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:50.574913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9db2b-7c57-4ac0-bf7a-c58e80d6bba5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:50.577647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1208/2389] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:50.581267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e0c9db2b-7c57-4ac0-bf7a-c58e80d6bba5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:50.581546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c0d6e-a4dc-418a-a018-ede5a2ac0dd0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:50.604354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c0d6e-a4dc-418a-a018-ede5a2ac0dd0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:50.627258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56715dfc-2941-4214-a660-c15ae8776609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:50.629905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56715dfc-2941-4214-a660-c15ae8776609 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:50.630443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56715dfc-2941-4214-a660-c15ae8776609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:50.630888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56715dfc-2941-4214-a660-c15ae8776609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:50.631701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56715dfc-2941-4214-a660-c15ae8776609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:50.636535 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56715dfc-2941-4214-a660-c15ae8776609 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:50.664677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1209/2390] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:50.666423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9db2b-7c57-4ac0-bf7a-c58e80d6bba5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:50.667713 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56715dfc-2941-4214-a660-c15ae8776609 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:50.668740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9db2b-7c57-4ac0-bf7a-c58e80d6bba5 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:50.680287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56715dfc-2941-4214-a660-c15ae8776609 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:50.680741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1182/2391] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:39:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:39:50.688720 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf793c2-88bd-443b-ac09-ec47e768fb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:50.689643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf793c2-88bd-443b-ac09-ec47e768fb5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:50.690068 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf793c2-88bd-443b-ac09-ec47e768fb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:50.690504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf793c2-88bd-443b-ac09-ec47e768fb5d admin admin] PATH_INFO: `/v3/projects/3b9fe768140a475384857c527694aa59/users/88365c5b477d49d38e13bc68d7bc78fc/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:50.738299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf793c2-88bd-443b-ac09-ec47e768fb5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9fe768140a475384857c527694aa59, user_id=88365c5b477d49d38e13bc68d7bc78fc, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:50.746759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf793c2-88bd-443b-ac09-ec47e768fb5d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:50.864945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1210/2392] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:50 2022] PUT /identity/v3/projects/3b9fe768140a475384857c527694aa59/users/88365c5b477d49d38e13bc68d7bc78fc/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:50.876656 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3f804-2542-4c36-99d7-4a201556f2e0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:50.876936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3f804-2542-4c36-99d7-4a201556f2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:50.877210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3f804-2542-4c36-99d7-4a201556f2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:50.958932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-12a3f804-2542-4c36-99d7-4a201556f2e0 None None] MFA Rules not processed for user `88365c5b477d49d38e13bc68d7bc78fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:50.963343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-12a3f804-2542-4c36-99d7-4a201556f2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:51.111139 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf793c2-88bd-443b-ac09-ec47e768fb5d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:51.113079 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1cf793c2-88bd-443b-ac09-ec47e768fb5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:51.181638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1183/2393] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:51.186463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3f804-2542-4c36-99d7-4a201556f2e0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:51.352872 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3f804-2542-4c36-99d7-4a201556f2e0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:51.359979 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf793c2-88bd-443b-ac09-ec47e768fb5d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:51.367627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:51.367627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:51.367627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:51.368365 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] PATH_INFO: `/v3/projects/1a5425bf731141088a5e23a4171918e4` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:51.369847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5425bf731141088a5e23a4171918e4)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:51.372048 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:51.380873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada09b91-18ed-4aca-8d8d-ee06cd83921e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:51.381514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada09b91-18ed-4aca-8d8d-ee06cd83921e admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:51.381803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada09b91-18ed-4aca-8d8d-ee06cd83921e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:51.382076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada09b91-18ed-4aca-8d8d-ee06cd83921e admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:51.383573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada09b91-18ed-4aca-8d8d-ee06cd83921e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:51.385940 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada09b91-18ed-4aca-8d8d-ee06cd83921e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:51.524818 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1184/2394] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:51 2022] POST /identity/v3/projects => generated 375 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:51.526319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada09b91-18ed-4aca-8d8d-ee06cd83921e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:51.547153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5425bf731141088a5e23a4171918e4', 'request_id': 'req-48be3037-454f-4214-acbc-91ce2cd9e369'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:51.557700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5425bf731141088a5e23a4171918e4', 'request_id': 'req-48be3037-454f-4214-acbc-91ce2cd9e369'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:51.655586 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada09b91-18ed-4aca-8d8d-ee06cd83921e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:51.672799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13ab76-7f5d-4c69-b8c9-5a64347b2cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:51.673438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13ab76-7f5d-4c69-b8c9-5a64347b2cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:51.673717 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13ab76-7f5d-4c69-b8c9-5a64347b2cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:51.673991 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13ab76-7f5d-4c69-b8c9-5a64347b2cb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:51.674627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13ab76-7f5d-4c69-b8c9-5a64347b2cb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:51.676718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13ab76-7f5d-4c69-b8c9-5a64347b2cb1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:51.677102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5425bf731141088a5e23a4171918e4', 'request_id': 'req-48be3037-454f-4214-acbc-91ce2cd9e369'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:51.691486 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:51.700685 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] The token cache is being invalidate because project 1a5425bf731141088a5e23a4171918e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:39:51.701505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:51.702862 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-48be3037-454f-4214-acbc-91ce2cd9e369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:39:51.707295 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1211/2395] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:39:51 2022] DELETE /identity/v3/projects/1a5425bf731141088a5e23a4171918e4 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:51.718834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c7c03-0a06-4d1b-be49-854f337664a6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:51.719369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c7c03-0a06-4d1b-be49-854f337664a6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:51.719802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c7c03-0a06-4d1b-be49-854f337664a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:51.774453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-481c7c03-0a06-4d1b-be49-854f337664a6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:51.775981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-481c7c03-0a06-4d1b-be49-854f337664a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:51.781125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1185/2396] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:51 2022] POST /identity/v3/users => generated 288 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:51.785087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13ab76-7f5d-4c69-b8c9-5a64347b2cb1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:51.787373 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8f13ab76-7f5d-4c69-b8c9-5a64347b2cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:51.806825 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1212/2397] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:51.815188 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afba4a8-fe1c-46f3-9f80-6ffd3d73f881 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:51.815471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afba4a8-fe1c-46f3-9f80-6ffd3d73f881 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:51.823034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afba4a8-fe1c-46f3-9f80-6ffd3d73f881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:51.848367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13ab76-7f5d-4c69-b8c9-5a64347b2cb1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:51.862476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2afba4a8-fe1c-46f3-9f80-6ffd3d73f881 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:51.864915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2afba4a8-fe1c-46f3-9f80-6ffd3d73f881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:51.869738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd37d2-f282-4616-a8cf-812d2b4da029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:51.876510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd37d2-f282-4616-a8cf-812d2b4da029 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:51.876510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd37d2-f282-4616-a8cf-812d2b4da029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:51.876510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd37d2-f282-4616-a8cf-812d2b4da029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:51.876510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd37d2-f282-4616-a8cf-812d2b4da029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:51.879418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1213/2398] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:51.881943 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd37d2-f282-4616-a8cf-812d2b4da029 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:51.887665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afba4a8-fe1c-46f3-9f80-6ffd3d73f881 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:51.911500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1186/2399] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:51.917555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd37d2-f282-4616-a8cf-812d2b4da029 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:51.928355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afba4a8-fe1c-46f3-9f80-6ffd3d73f881 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:51.941167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd37d2-f282-4616-a8cf-812d2b4da029 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:51.942088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57307f-bb03-45bc-9b7e-4dd2eb110771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:51.942922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57307f-bb03-45bc-9b7e-4dd2eb110771 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:51.943306 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57307f-bb03-45bc-9b7e-4dd2eb110771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:51.944458 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57307f-bb03-45bc-9b7e-4dd2eb110771 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:51.945781 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57307f-bb03-45bc-9b7e-4dd2eb110771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:39:51.946300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57307f-bb03-45bc-9b7e-4dd2eb110771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:51.948053 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57307f-bb03-45bc-9b7e-4dd2eb110771 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:51.952758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323a0a1-21bf-4db9-8ed5-d9a5a129b79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:51.967850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a0a1-21bf-4db9-8ed5-d9a5a129b79f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:51.968412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a0a1-21bf-4db9-8ed5-d9a5a129b79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:51.968836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a0a1-21bf-4db9-8ed5-d9a5a129b79f admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:51.969817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323a0a1-21bf-4db9-8ed5-d9a5a129b79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:51.971800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323a0a1-21bf-4db9-8ed5-d9a5a129b79f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:51.986098 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1214/2400] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:39:51.990142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57307f-bb03-45bc-9b7e-4dd2eb110771 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:52.000791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1187/2401] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:52.020174 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323a0a1-21bf-4db9-8ed5-d9a5a129b79f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:52.022147 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57307f-bb03-45bc-9b7e-4dd2eb110771 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:52.032269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46b731-8e37-4812-81cd-45f9004fe1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:52.032269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46b731-8e37-4812-81cd-45f9004fe1bb admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:52.032269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46b731-8e37-4812-81cd-45f9004fe1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:52.032269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46b731-8e37-4812-81cd-45f9004fe1bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:52.087232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46b731-8e37-4812-81cd-45f9004fe1bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:52.087232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46b731-8e37-4812-81cd-45f9004fe1bb admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:52.088333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323a0a1-21bf-4db9-8ed5-d9a5a129b79f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:52.101851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a504d-4df9-4cf4-b9c4-ef1468e8f8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:52.102536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a504d-4df9-4cf4-b9c4-ef1468e8f8d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:52.102906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a504d-4df9-4cf4-b9c4-ef1468e8f8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:52.103230 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a504d-4df9-4cf4-b9c4-ef1468e8f8d9 admin admin] PATH_INFO: `/v3/projects/f54ced09c9c34af7a281017705b48a6a/users/8be58f1a8db540088dbc5294ffbd6f82/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:52.159078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a504d-4df9-4cf4-b9c4-ef1468e8f8d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54ced09c9c34af7a281017705b48a6a, user_id=8be58f1a8db540088dbc5294ffbd6f82, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:52.161858 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a504d-4df9-4cf4-b9c4-ef1468e8f8d9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:52.677662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1215/2402] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:51 2022] POST /identity/v3/projects => generated 377 bytes in 688 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:52.679140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46b731-8e37-4812-81cd-45f9004fe1bb admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:52.716744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1188/2403] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:52 2022] PUT /identity/v3/projects/f54ced09c9c34af7a281017705b48a6a/users/8be58f1a8db540088dbc5294ffbd6f82/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:52.727706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a33d3-f916-42fb-b37d-31dd88a7f956 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:52.728324 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a33d3-f916-42fb-b37d-31dd88a7f956 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:52.728888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a33d3-f916-42fb-b37d-31dd88a7f956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:52.792032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:52.814890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-531a33d3-f916-42fb-b37d-31dd88a7f956 None None] MFA Rules not processed for user `8be58f1a8db540088dbc5294ffbd6f82`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:52.824477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-531a33d3-f916-42fb-b37d-31dd88a7f956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:52.902444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46b731-8e37-4812-81cd-45f9004fe1bb admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:52.930665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1189/2404] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:52.936346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a33d3-f916-42fb-b37d-31dd88a7f956 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:53.043050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315356ce-799b-4815-9bf7-8afa9dd0e546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:53.043050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315356ce-799b-4815-9bf7-8afa9dd0e546 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:53.043050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315356ce-799b-4815-9bf7-8afa9dd0e546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:53.043050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315356ce-799b-4815-9bf7-8afa9dd0e546 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:53.043467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315356ce-799b-4815-9bf7-8afa9dd0e546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:53.043467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315356ce-799b-4815-9bf7-8afa9dd0e546 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:53.109239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a33d3-f916-42fb-b37d-31dd88a7f956 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:53.123642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de28ab-30c6-4695-ab4d-6899670b3d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:53.124444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de28ab-30c6-4695-ab4d-6899670b3d28 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:53.124835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de28ab-30c6-4695-ab4d-6899670b3d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:53.126501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de28ab-30c6-4695-ab4d-6899670b3d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:53.127526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de28ab-30c6-4695-ab4d-6899670b3d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:53.130637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de28ab-30c6-4695-ab4d-6899670b3d28 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:53.187576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1216/2405] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:52 2022] POST /identity/v3/users => generated 289 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:53.201655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315356ce-799b-4815-9bf7-8afa9dd0e546 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:53.232096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315356ce-799b-4815-9bf7-8afa9dd0e546 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:53.250222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cd270-2060-47b1-85ff-837a7b7d9e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:53.254769 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cd270-2060-47b1-85ff-837a7b7d9e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:53.255062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cd270-2060-47b1-85ff-837a7b7d9e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:53.258274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cd270-2060-47b1-85ff-837a7b7d9e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:53.259297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85cd270-2060-47b1-85ff-837a7b7d9e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:53.271700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85cd270-2060-47b1-85ff-837a7b7d9e16 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:53.277885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1190/2406] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:52 2022] POST /identity/v3/projects => generated 375 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:53.289479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de28ab-30c6-4695-ab4d-6899670b3d28 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:53.308829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1217/2407] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:53.318549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cd270-2060-47b1-85ff-837a7b7d9e16 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:53.341462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cd270-2060-47b1-85ff-837a7b7d9e16 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:53.466633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de28ab-30c6-4695-ab4d-6899670b3d28 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:53.480088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11863d26-0199-4b2d-9e29-b12f9bab5600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:53.480795 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11863d26-0199-4b2d-9e29-b12f9bab5600 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:53.481385 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11863d26-0199-4b2d-9e29-b12f9bab5600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:53.483047 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11863d26-0199-4b2d-9e29-b12f9bab5600 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:53.483822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11863d26-0199-4b2d-9e29-b12f9bab5600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:53.486336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11863d26-0199-4b2d-9e29-b12f9bab5600 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:53.501841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07dc2dd-da47-4815-8acb-47630c337d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:53.502845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dc2dd-da47-4815-8acb-47630c337d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:53.503122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dc2dd-da47-4815-8acb-47630c337d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:53.503502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dc2dd-da47-4815-8acb-47630c337d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:53.504529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07dc2dd-da47-4815-8acb-47630c337d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:53.506423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07dc2dd-da47-4815-8acb-47630c337d67 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:53.526640 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1218/2408] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:39:53.534629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07dc2dd-da47-4815-8acb-47630c337d67 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:53.554629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07dc2dd-da47-4815-8acb-47630c337d67 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:53.576070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d80d0-4e2f-45bf-921f-383ae81f89e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:53.576070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d80d0-4e2f-45bf-921f-383ae81f89e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:53.576070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d80d0-4e2f-45bf-921f-383ae81f89e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:53.576070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d80d0-4e2f-45bf-921f-383ae81f89e9 admin admin] PATH_INFO: `/v3/projects/7a32cf48d21c4ec9b39b3f5fa2ee752a/users/7072bd179b3d4ec2bcc5fcd030c3df59/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:53.591810 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1191/2409] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:53 2022] POST /identity/v3/users => generated 294 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:53.594630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11863d26-0199-4b2d-9e29-b12f9bab5600 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:53.611838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11863d26-0199-4b2d-9e29-b12f9bab5600 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:53.616519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d80d0-4e2f-45bf-921f-383ae81f89e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a32cf48d21c4ec9b39b3f5fa2ee752a, user_id=7072bd179b3d4ec2bcc5fcd030c3df59, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:53.620447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb048538-92e8-4752-b6d0-ccfad22f5267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:53.622007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb048538-92e8-4752-b6d0-ccfad22f5267 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:53.622638 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb048538-92e8-4752-b6d0-ccfad22f5267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:53.622926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d80d0-4e2f-45bf-921f-383ae81f89e9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:53.623368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb048538-92e8-4752-b6d0-ccfad22f5267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:53.624564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb048538-92e8-4752-b6d0-ccfad22f5267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:53.627204 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb048538-92e8-4752-b6d0-ccfad22f5267 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:53.643512 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1192/2410] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:53.652251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb048538-92e8-4752-b6d0-ccfad22f5267 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:53.669498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb048538-92e8-4752-b6d0-ccfad22f5267 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:53.678528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859a39f-6e38-4ba3-9eab-92179110b9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:53.678528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859a39f-6e38-4ba3-9eab-92179110b9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:53.678528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859a39f-6e38-4ba3-9eab-92179110b9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:53.679131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859a39f-6e38-4ba3-9eab-92179110b9d7 admin admin] PATH_INFO: `/v3/projects/a3bc9a1c1d8f41b5a5eb9055276b9cff/users/40a502a082624984b003aeb836862bab/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:53.704928 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1219/2411] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:53 2022] PUT /identity/v3/projects/7a32cf48d21c4ec9b39b3f5fa2ee752a/users/7072bd179b3d4ec2bcc5fcd030c3df59/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:53.714843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859a39f-6e38-4ba3-9eab-92179110b9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3bc9a1c1d8f41b5a5eb9055276b9cff, user_id=40a502a082624984b003aeb836862bab, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:53.716115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81bbe5-4bc0-4781-bad7-b1c3643cc48d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:53.716314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81bbe5-4bc0-4781-bad7-b1c3643cc48d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:53.716600 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81bbe5-4bc0-4781-bad7-b1c3643cc48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:53.717670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859a39f-6e38-4ba3-9eab-92179110b9d7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:53.758454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:53.762714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-da81bbe5-4bc0-4781-bad7-b1c3643cc48d None None] MFA Rules not processed for user `7072bd179b3d4ec2bcc5fcd030c3df59`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:53.767335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-da81bbe5-4bc0-4781-bad7-b1c3643cc48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:53.823867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1193/2412] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:53 2022] PUT /identity/v3/projects/a3bc9a1c1d8f41b5a5eb9055276b9cff/users/40a502a082624984b003aeb836862bab/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:53.829171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859a39f-6e38-4ba3-9eab-92179110b9d7 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:53.916824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1220/2413] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:39:53.930298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81bbe5-4bc0-4781-bad7-b1c3643cc48d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:53.932098 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-da81bbe5-4bc0-4781-bad7-b1c3643cc48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:53.985653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859a39f-6e38-4ba3-9eab-92179110b9d7 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:53.991179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793c8c-45f6-4b4d-9a82-586870bd0628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:53.992229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793c8c-45f6-4b4d-9a82-586870bd0628 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:53.992580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793c8c-45f6-4b4d-9a82-586870bd0628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:53.992933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793c8c-45f6-4b4d-9a82-586870bd0628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:53.993663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a793c8c-45f6-4b4d-9a82-586870bd0628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:53.995486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a793c8c-45f6-4b4d-9a82-586870bd0628 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:54.007976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1194/2414] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:54.012338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793c8c-45f6-4b4d-9a82-586870bd0628 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:54.024642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793c8c-45f6-4b4d-9a82-586870bd0628 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:54.032052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fb856-800d-4bf7-89bc-4db44efab34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:54.035731 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fb856-800d-4bf7-89bc-4db44efab34e admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:54.035731 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fb856-800d-4bf7-89bc-4db44efab34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:54.035731 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fb856-800d-4bf7-89bc-4db44efab34e admin admin] PATH_INFO: `/v3/projects/a3bc9a1c1d8f41b5a5eb9055276b9cff/users/40a502a082624984b003aeb836862bab/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:54.036323 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fb856-800d-4bf7-89bc-4db44efab34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3bc9a1c1d8f41b5a5eb9055276b9cff, user_id=40a502a082624984b003aeb836862bab, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:54.039880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fb856-800d-4bf7-89bc-4db44efab34e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:54.057925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1195/2415] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:54 2022] PUT /identity/v3/projects/a3bc9a1c1d8f41b5a5eb9055276b9cff/users/40a502a082624984b003aeb836862bab/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:54.058620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81bbe5-4bc0-4781-bad7-b1c3643cc48d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:54.068607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6a3df-2f8d-435c-a040-2798706e81d4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:54.069027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6a3df-2f8d-435c-a040-2798706e81d4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:54.069372 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6a3df-2f8d-435c-a040-2798706e81d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:54.111411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-51e6a3df-2f8d-435c-a040-2798706e81d4 None None] MFA Rules not processed for user `40a502a082624984b003aeb836862bab`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:54.112793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-51e6a3df-2f8d-435c-a040-2798706e81d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:54.128590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18c7c7-e706-4f86-9e6a-b25c58b9b1c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:54.129286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18c7c7-e706-4f86-9e6a-b25c58b9b1c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:54.129567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18c7c7-e706-4f86-9e6a-b25c58b9b1c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:54.129837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18c7c7-e706-4f86-9e6a-b25c58b9b1c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:54.226734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:54.246349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1196/2416] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:54.258028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18c7c7-e706-4f86-9e6a-b25c58b9b1c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:54.261059 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6a3df-2f8d-435c-a040-2798706e81d4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:54.261626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18c7c7-e706-4f86-9e6a-b25c58b9b1c9 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:54.263029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-51e6a3df-2f8d-435c-a040-2798706e81d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:54.281313 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6a3df-2f8d-435c-a040-2798706e81d4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:54.286238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeb10f-e668-4566-8eb1-1ed8f9eda81e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:54.287062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeb10f-e668-4566-8eb1-1ed8f9eda81e service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:54.287514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeb10f-e668-4566-8eb1-1ed8f9eda81e service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:54.287949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeb10f-e668-4566-8eb1-1ed8f9eda81e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:54.290715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0afeb10f-e668-4566-8eb1-1ed8f9eda81e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:54.291794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1221/2417] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:39:54.421058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afeb10f-e668-4566-8eb1-1ed8f9eda81e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:54.424046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afeb10f-e668-4566-8eb1-1ed8f9eda81e service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:54.443719 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1197/2418] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:39:54.822105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:54.957119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:55.889625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:56.451822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712dc24-48bb-45d5-999d-285b47595232 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:56.452920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712dc24-48bb-45d5-999d-285b47595232 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:56.453362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712dc24-48bb-45d5-999d-285b47595232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:56.527162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8712dc24-48bb-45d5-999d-285b47595232 None None] MFA Rules not processed for user `394c67bcc3864bb2aca2f4faec834029`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:39:56.528763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8712dc24-48bb-45d5-999d-285b47595232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:56.538787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:56.610184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1222/2419] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:39:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:39:56.614629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeb10f-e668-4566-8eb1-1ed8f9eda81e service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:56.616208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0afeb10f-e668-4566-8eb1-1ed8f9eda81e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:39:56.685655 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeb10f-e668-4566-8eb1-1ed8f9eda81e service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:56.722411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1ab94-ab35-4e78-a391-5de11c6af582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:56.722411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1ab94-ab35-4e78-a391-5de11c6af582 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:56.722411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1ab94-ab35-4e78-a391-5de11c6af582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:56.722411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1ab94-ab35-4e78-a391-5de11c6af582 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:56.723132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1ab94-ab35-4e78-a391-5de11c6af582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:56.723132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1ab94-ab35-4e78-a391-5de11c6af582 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:57.030939 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1198/2420] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:39:56 2022] POST /identity/v3/projects => generated 375 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:39:57.037293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712dc24-48bb-45d5-999d-285b47595232 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:57.057491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712dc24-48bb-45d5-999d-285b47595232 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:57.206145 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b8ab6-9d62-4e25-9333-b82ac062a39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:57.206986 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b8ab6-9d62-4e25-9333-b82ac062a39e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:57.207432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b8ab6-9d62-4e25-9333-b82ac062a39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:57.207901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b8ab6-9d62-4e25-9333-b82ac062a39e admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:57.208785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b8ab6-9d62-4e25-9333-b82ac062a39e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:57.211959 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b8ab6-9d62-4e25-9333-b82ac062a39e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:57.297869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1ab94-ab35-4e78-a391-5de11c6af582 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:57.310398 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1223/2421] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:39:57 2022] POST /identity/v3/users => generated 294 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:39:57.324839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b8ab6-9d62-4e25-9333-b82ac062a39e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:57.340233 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b8ab6-9d62-4e25-9333-b82ac062a39e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:57.346640 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80ef7f-d807-44f1-9934-e06b939aabd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:57.347311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80ef7f-d807-44f1-9934-e06b939aabd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:57.347641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80ef7f-d807-44f1-9934-e06b939aabd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:57.347919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80ef7f-d807-44f1-9934-e06b939aabd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:57.348667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80ef7f-d807-44f1-9934-e06b939aabd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:57.353238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80ef7f-d807-44f1-9934-e06b939aabd2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:57.400093 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1224/2422] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:57.410111 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80ef7f-d807-44f1-9934-e06b939aabd2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:57.442484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80ef7f-d807-44f1-9934-e06b939aabd2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:57.461947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa6299-dc89-4372-9c88-c4ab1d10a39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:57.464134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa6299-dc89-4372-9c88-c4ab1d10a39d admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:57.470800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa6299-dc89-4372-9c88-c4ab1d10a39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:57.470800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa6299-dc89-4372-9c88-c4ab1d10a39d admin admin] PATH_INFO: `/v3/projects/4842e6e3e892400cbccac28d678c8190/users/2aac329bbae0423faabd3fc4601c859d/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:57.471041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1ab94-ab35-4e78-a391-5de11c6af582 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:57.479646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c435e9-262d-4385-8f1b-8856594a06ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:57.480418 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c435e9-262d-4385-8f1b-8856594a06ff service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:57.481820 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c435e9-262d-4385-8f1b-8856594a06ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:57.482094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c435e9-262d-4385-8f1b-8856594a06ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:57.504306 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa6299-dc89-4372-9c88-c4ab1d10a39d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4842e6e3e892400cbccac28d678c8190, user_id=2aac329bbae0423faabd3fc4601c859d, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:57.513271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa6299-dc89-4372-9c88-c4ab1d10a39d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:57.606001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1225/2423] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:57 2022] PUT /identity/v3/projects/4842e6e3e892400cbccac28d678c8190/users/2aac329bbae0423faabd3fc4601c859d/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:39:57.621824 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa6299-dc89-4372-9c88-c4ab1d10a39d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:57.621980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:57.637028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c435e9-262d-4385-8f1b-8856594a06ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:57.641670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c435e9-262d-4385-8f1b-8856594a06ff service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:57.661627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1199/2424] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:39:57.766711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa6299-dc89-4372-9c88-c4ab1d10a39d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:57.788984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e5837-3789-4512-84d9-294030f3d558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:57.790056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e5837-3789-4512-84d9-294030f3d558 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:57.790511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e5837-3789-4512-84d9-294030f3d558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:57.790943 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e5837-3789-4512-84d9-294030f3d558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:57.796849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e5837-3789-4512-84d9-294030f3d558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:57.798898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e5837-3789-4512-84d9-294030f3d558 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:57.804806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:57.816108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1226/2425] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:39:57.820287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c435e9-262d-4385-8f1b-8856594a06ff service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:39:57.845127 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c435e9-262d-4385-8f1b-8856594a06ff service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:39:57.872760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c55c89-a8da-4632-9660-8fd9046101e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:39:57.873587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55c89-a8da-4632-9660-8fd9046101e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:39:57.873872 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55c89-a8da-4632-9660-8fd9046101e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:39:57.874144 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55c89-a8da-4632-9660-8fd9046101e3 admin admin] PATH_INFO: `/v3/projects/4842e6e3e892400cbccac28d678c8190/users/2aac329bbae0423faabd3fc4601c859d/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:39:57.887056 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c55c89-a8da-4632-9660-8fd9046101e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4842e6e3e892400cbccac28d678c8190, user_id=2aac329bbae0423faabd3fc4601c859d, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:39:57.890101 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c55c89-a8da-4632-9660-8fd9046101e3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:39:57.918683 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1200/2426] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:39:57 2022] PUT /identity/v3/projects/4842e6e3e892400cbccac28d678c8190/users/2aac329bbae0423faabd3fc4601c859d/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:39:58.095200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:39:59.970241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e5837-3789-4512-84d9-294030f3d558 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:00.070225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e5837-3789-4512-84d9-294030f3d558 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:00.187747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97839f00-a139-4d00-a107-929ff9d6fc53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:00.188420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97839f00-a139-4d00-a107-929ff9d6fc53 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:00.188689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97839f00-a139-4d00-a107-929ff9d6fc53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:00.189047 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97839f00-a139-4d00-a107-929ff9d6fc53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:00.338161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97839f00-a139-4d00-a107-929ff9d6fc53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:00.338872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97839f00-a139-4d00-a107-929ff9d6fc53 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:00.354405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1227/2427] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:39:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:07.078553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ad0d3-c928-4201-945f-4ca149bbdb78 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:07.079591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ad0d3-c928-4201-945f-4ca149bbdb78 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:07.080346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ad0d3-c928-4201-945f-4ca149bbdb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:07.148046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ae5ad0d3-c928-4201-945f-4ca149bbdb78 None None] MFA Rules not processed for user `2aac329bbae0423faabd3fc4601c859d`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:07.149323 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ae5ad0d3-c928-4201-945f-4ca149bbdb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:07.349496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97839f00-a139-4d00-a107-929ff9d6fc53 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:07.351409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1201/2428] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:07.352129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-97839f00-a139-4d00-a107-929ff9d6fc53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:07.520542 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:07.531215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97839f00-a139-4d00-a107-929ff9d6fc53 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:07.539838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c9f64-e34e-45c5-b62e-1ad0de29dbc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:07.540500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c9f64-e34e-45c5-b62e-1ad0de29dbc0 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:07.540785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c9f64-e34e-45c5-b62e-1ad0de29dbc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:07.541064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c9f64-e34e-45c5-b62e-1ad0de29dbc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:07.639627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c9f64-e34e-45c5-b62e-1ad0de29dbc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:07.641439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c9f64-e34e-45c5-b62e-1ad0de29dbc0 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:07.655475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1228/2429] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:40:10.520611 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ad0d3-c928-4201-945f-4ca149bbdb78 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:10.524649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ae5ad0d3-c928-4201-945f-4ca149bbdb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:10.782990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ad0d3-c928-4201-945f-4ca149bbdb78 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:10.789237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:10.790140 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:10.790519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:10.791023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] PATH_INFO: `/v3/users/8b713f6d89cd46b9a1d8d932ea6288aa` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:10.793181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b713f6d89cd46b9a1d8d932ea6288aa)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:10.795320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:10.798269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:11.099411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b713f6d89cd46b9a1d8d932ea6288aa'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:11.191184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b713f6d89cd46b9a1d8d932ea6288aa'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:11.204994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b713f6d89cd46b9a1d8d932ea6288aa'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:11.205754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b713f6d89cd46b9a1d8d932ea6288aa'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:11.206716 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b713f6d89cd46b9a1d8d932ea6288aa'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:11.227214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1202/2430] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:40:10 2022] DELETE /identity/v3/users/8b713f6d89cd46b9a1d8d932ea6288aa => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:11.237730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cfa6c-c5e5-40b9-92ef-5a797adf11bb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:11.238261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cfa6c-c5e5-40b9-92ef-5a797adf11bb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:11.240631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cfa6c-c5e5-40b9-92ef-5a797adf11bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:11.290857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bc1cfa6c-c5e5-40b9-92ef-5a797adf11bb None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:11.292637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bc1cfa6c-c5e5-40b9-92ef-5a797adf11bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:11.407383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:11.408925 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:11.422348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1229/2431] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:11.433967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bca03c-5a21-40f7-941c-0f3291dab63c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:11.438427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cfa6c-c5e5-40b9-92ef-5a797adf11bb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:11.439239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:11.440265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bc1cfa6c-c5e5-40b9-92ef-5a797adf11bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:11.440504 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:11.441112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:11.441602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] PATH_INFO: `/v3/users/88365c5b477d49d38e13bc68d7bc78fc` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:11.443107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88365c5b477d49d38e13bc68d7bc78fc)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:11.446906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:11.565546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cfa6c-c5e5-40b9-92ef-5a797adf11bb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:11.570320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b05cbb-016b-4b3e-9ca9-89b6ccfe03dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:11.571024 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b05cbb-016b-4b3e-9ca9-89b6ccfe03dc service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:11.571371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b05cbb-016b-4b3e-9ca9-89b6ccfe03dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:11.571707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b05cbb-016b-4b3e-9ca9-89b6ccfe03dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:11.581575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b05cbb-016b-4b3e-9ca9-89b6ccfe03dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:11.583386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b05cbb-016b-4b3e-9ca9-89b6ccfe03dc service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:11.594679 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1230/2432] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:11.689239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88365c5b477d49d38e13bc68d7bc78fc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:11.762772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88365c5b477d49d38e13bc68d7bc78fc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:11.784425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88365c5b477d49d38e13bc68d7bc78fc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:11.785342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88365c5b477d49d38e13bc68d7bc78fc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:11.786119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88365c5b477d49d38e13bc68d7bc78fc'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:11.817173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1203/2433] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:40:11 2022] DELETE /identity/v3/users/88365c5b477d49d38e13bc68d7bc78fc => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:11.824055 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b665e67-1430-418b-8cb1-16b7f7f62961 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:11.824357 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b665e67-1430-418b-8cb1-16b7f7f62961 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:11.824638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b665e67-1430-418b-8cb1-16b7f7f62961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:11.877907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0b665e67-1430-418b-8cb1-16b7f7f62961 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:11.880803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0b665e67-1430-418b-8cb1-16b7f7f62961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:11.989800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:12.026669 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1231/2434] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:12.037584 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:12.041003 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:12.146811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdba11b-26bd-412d-8e6e-819988c43506 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:12.157615 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e108a-5844-4c08-a5be-f2c8889090fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:12.158090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e108a-5844-4c08-a5be-f2c8889090fc service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:12.158376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e108a-5844-4c08-a5be-f2c8889090fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:12.158649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e108a-5844-4c08-a5be-f2c8889090fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:12.171860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:12.176894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e108a-5844-4c08-a5be-f2c8889090fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:12.178558 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e108a-5844-4c08-a5be-f2c8889090fc service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:12.196504 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1204/2435] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:12.417425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b665e67-1430-418b-8cb1-16b7f7f62961 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:12.417425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0b665e67-1430-418b-8cb1-16b7f7f62961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:12.429661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b665e67-1430-418b-8cb1-16b7f7f62961 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:12.437453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:12.440303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:12.440303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:12.440303 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] PATH_INFO: `/v3/projects/fa9cc9f634d34119868bda2ce11378af` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:12.442369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa9cc9f634d34119868bda2ce11378af)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:12.445655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:12.659830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa9cc9f634d34119868bda2ce11378af', 'request_id': 'req-903b1430-84ea-4ebb-a176-37ead175c5c9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:12.684914 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa9cc9f634d34119868bda2ce11378af', 'request_id': 'req-903b1430-84ea-4ebb-a176-37ead175c5c9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:12.780722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa9cc9f634d34119868bda2ce11378af', 'request_id': 'req-903b1430-84ea-4ebb-a176-37ead175c5c9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:12.785503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e108a-5844-4c08-a5be-f2c8889090fc service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:12.786868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bd4e108a-5844-4c08-a5be-f2c8889090fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:12.801368 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] The token cache is being invalidate because project fa9cc9f634d34119868bda2ce11378af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:40:12.801667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:12.805150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-903b1430-84ea-4ebb-a176-37ead175c5c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:12.809595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1232/2436] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:40:12 2022] DELETE /identity/v3/projects/fa9cc9f634d34119868bda2ce11378af => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:12.817508 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171c446-0d9f-424c-aaa2-914621221141 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:12.817796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171c446-0d9f-424c-aaa2-914621221141 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:12.818080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171c446-0d9f-424c-aaa2-914621221141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:12.837894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e108a-5844-4c08-a5be-f2c8889090fc service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:12.877212 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d171c446-0d9f-424c-aaa2-914621221141 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:12.882916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d171c446-0d9f-424c-aaa2-914621221141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:12.991776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:12.992560 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:12.992948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:12.993321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] PATH_INFO: `/v3/projects/3b9fe768140a475384857c527694aa59` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:12.995123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b9fe768140a475384857c527694aa59)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:12.997743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:13.042613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1233/2437] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:13.044882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fe292-4dec-47ef-beee-e95eedcf6485 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:13.045176 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fe292-4dec-47ef-beee-e95eedcf6485 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:13.045467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fe292-4dec-47ef-beee-e95eedcf6485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:13.081829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-df3fe292-4dec-47ef-beee-e95eedcf6485 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:13.083264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-df3fe292-4dec-47ef-beee-e95eedcf6485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:13.092218 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1234/2438] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:13.096910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fe292-4dec-47ef-beee-e95eedcf6485 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:13.108468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fe292-4dec-47ef-beee-e95eedcf6485 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:13.113571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864532d-f44e-4145-a641-767145db09e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:13.114195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864532d-f44e-4145-a641-767145db09e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:13.114553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864532d-f44e-4145-a641-767145db09e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:13.114810 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864532d-f44e-4145-a641-767145db09e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:13.115618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864532d-f44e-4145-a641-767145db09e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:40:13.116157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864532d-f44e-4145-a641-767145db09e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:13.126307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864532d-f44e-4145-a641-767145db09e2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:13.131730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1235/2439] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:40:13.136464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864532d-f44e-4145-a641-767145db09e2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:13.148434 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864532d-f44e-4145-a641-767145db09e2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:13.153776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d017376-529c-4106-8554-200a2fc33c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:13.154595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d017376-529c-4106-8554-200a2fc33c69 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:13.155091 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d017376-529c-4106-8554-200a2fc33c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:13.155363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d017376-529c-4106-8554-200a2fc33c69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:13.155997 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d017376-529c-4106-8554-200a2fc33c69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:13.158015 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d017376-529c-4106-8554-200a2fc33c69 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:13.197122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b9fe768140a475384857c527694aa59', 'request_id': 'req-3f07e7fe-d83d-4641-a3cd-55c31c923816'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:13.212399 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b9fe768140a475384857c527694aa59', 'request_id': 'req-3f07e7fe-d83d-4641-a3cd-55c31c923816'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:13.328487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b9fe768140a475384857c527694aa59', 'request_id': 'req-3f07e7fe-d83d-4641-a3cd-55c31c923816'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:13.337439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1236/2440] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:40:13 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:40:13.341759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d017376-529c-4106-8554-200a2fc33c69 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:13.345773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] The token cache is being invalidate because project 3b9fe768140a475384857c527694aa59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:40:13.346174 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:13.347167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:13.349523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1205/2441] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:40:12 2022] DELETE /identity/v3/projects/3b9fe768140a475384857c527694aa59 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:13.355845 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:13.357430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:13.385898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:13.433823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d017376-529c-4106-8554-200a2fc33c69 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:13.440762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce986fc-28fe-4364-bfac-568dba6708aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:13.441485 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce986fc-28fe-4364-bfac-568dba6708aa admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:13.441761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce986fc-28fe-4364-bfac-568dba6708aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:13.442030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce986fc-28fe-4364-bfac-568dba6708aa admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:13.442787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce986fc-28fe-4364-bfac-568dba6708aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:13.445661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce986fc-28fe-4364-bfac-568dba6708aa admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:13.466506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07e7fe-d83d-4641-a3cd-55c31c923816 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:13.471792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:13.473318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:13.473318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:13.473318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] PATH_INFO: `/v3/users/8be58f1a8db540088dbc5294ffbd6f82` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:13.474513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be58f1a8db540088dbc5294ffbd6f82)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:13.477635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:13.540356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1237/2442] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:40:13 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:40:13.544137 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce986fc-28fe-4364-bfac-568dba6708aa admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:13.545921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1ce986fc-28fe-4364-bfac-568dba6708aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:13.561146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce986fc-28fe-4364-bfac-568dba6708aa admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:13.568777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeccce7-a007-42db-811b-4ae586bdbba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:13.572220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeccce7-a007-42db-811b-4ae586bdbba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:13.572629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeccce7-a007-42db-811b-4ae586bdbba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:13.572992 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeccce7-a007-42db-811b-4ae586bdbba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:13.573752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeccce7-a007-42db-811b-4ae586bdbba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:13.575518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeccce7-a007-42db-811b-4ae586bdbba7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:13.591643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1238/2443] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:40:13.593038 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeccce7-a007-42db-811b-4ae586bdbba7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:13.605144 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeccce7-a007-42db-811b-4ae586bdbba7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:13.609663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:13.612941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecfca2-9b47-4dd1-af31-f7882f76ef2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:13.618677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecfca2-9b47-4dd1-af31-f7882f76ef2b admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:13.618677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecfca2-9b47-4dd1-af31-f7882f76ef2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:13.618677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecfca2-9b47-4dd1-af31-f7882f76ef2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:13.618677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ecfca2-9b47-4dd1-af31-f7882f76ef2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:13.618677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ecfca2-9b47-4dd1-af31-f7882f76ef2b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:13.631180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1239/2444] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:40:13.632637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecfca2-9b47-4dd1-af31-f7882f76ef2b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:13.653874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecfca2-9b47-4dd1-af31-f7882f76ef2b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:13.674159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66660277-97cc-4437-bc35-c74d1ccd18fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:13.676357 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66660277-97cc-4437-bc35-c74d1ccd18fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:13.676357 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66660277-97cc-4437-bc35-c74d1ccd18fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:13.676357 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66660277-97cc-4437-bc35-c74d1ccd18fd admin admin] PATH_INFO: `/v3/projects/097fb8408bfb479d87e7516a93970e61/users/d7f2dbb97068490d924ae132224dc8ff/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:13.711032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66660277-97cc-4437-bc35-c74d1ccd18fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=097fb8408bfb479d87e7516a93970e61, user_id=d7f2dbb97068490d924ae132224dc8ff, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:13.720160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66660277-97cc-4437-bc35-c74d1ccd18fd admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:13.724127 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be58f1a8db540088dbc5294ffbd6f82'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:13.792184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1240/2445] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:40:13 2022] PUT /identity/v3/projects/097fb8408bfb479d87e7516a93970e61/users/d7f2dbb97068490d924ae132224dc8ff/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:13.798747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea11504-68da-447d-8798-e308aa41bff9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:13.799030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea11504-68da-447d-8798-e308aa41bff9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:13.799310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea11504-68da-447d-8798-e308aa41bff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:13.809193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be58f1a8db540088dbc5294ffbd6f82'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:13.829756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be58f1a8db540088dbc5294ffbd6f82'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:13.831239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be58f1a8db540088dbc5294ffbd6f82'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:13.832743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be58f1a8db540088dbc5294ffbd6f82'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:13.849069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0ea11504-68da-447d-8798-e308aa41bff9 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:13.850743 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0ea11504-68da-447d-8798-e308aa41bff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:13.858751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1206/2446] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:40:13 2022] DELETE /identity/v3/users/8be58f1a8db540088dbc5294ffbd6f82 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:13.969793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1241/2447] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:13.980743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:13.982358 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:14.047031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441fc5b-14ea-4a98-bdee-b2ef0f549016 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:14.050655 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c085e6-90f1-4573-96a3-7d4cef8c4b4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:14.051251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c085e6-90f1-4573-96a3-7d4cef8c4b4d service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:14.051527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c085e6-90f1-4573-96a3-7d4cef8c4b4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:14.051798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c085e6-90f1-4573-96a3-7d4cef8c4b4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:14.118227 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c085e6-90f1-4573-96a3-7d4cef8c4b4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:14.119710 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c085e6-90f1-4573-96a3-7d4cef8c4b4d service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:14.131773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1207/2448] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:40:14.414649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea11504-68da-447d-8798-e308aa41bff9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:14.416183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0ea11504-68da-447d-8798-e308aa41bff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:14.430491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea11504-68da-447d-8798-e308aa41bff9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:14.438972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:14.440150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:14.440150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:14.440444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] PATH_INFO: `/v3/projects/f54ced09c9c34af7a281017705b48a6a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:14.441856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f54ced09c9c34af7a281017705b48a6a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:14.444177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:14.576311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f54ced09c9c34af7a281017705b48a6a', 'request_id': 'req-2da82af1-0e97-4de1-b76a-10e1f0ae6349'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:14.582842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f54ced09c9c34af7a281017705b48a6a', 'request_id': 'req-2da82af1-0e97-4de1-b76a-10e1f0ae6349'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:14.671204 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f54ced09c9c34af7a281017705b48a6a', 'request_id': 'req-2da82af1-0e97-4de1-b76a-10e1f0ae6349'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:14.685606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] The token cache is being invalidate because project f54ced09c9c34af7a281017705b48a6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:40:14.685900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:14.686586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:14.688276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1242/2449] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:40:14 2022] DELETE /identity/v3/projects/f54ced09c9c34af7a281017705b48a6a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:14.689760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:14.692191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c085e6-90f1-4573-96a3-7d4cef8c4b4d service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:14.703891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c085e6-90f1-4573-96a3-7d4cef8c4b4d service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:14.769355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:14.774303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e375f6-0ca6-4876-a130-7d4285fd62ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:14.781029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e375f6-0ca6-4876-a130-7d4285fd62ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:14.781029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e375f6-0ca6-4876-a130-7d4285fd62ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:14.781029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e375f6-0ca6-4876-a130-7d4285fd62ea admin admin] PATH_INFO: `/v3/users/40a502a082624984b003aeb836862bab` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:14.781029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e375f6-0ca6-4876-a130-7d4285fd62ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=40a502a082624984b003aeb836862bab)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:14.781029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e375f6-0ca6-4876-a130-7d4285fd62ea admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:15.042078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-16e375f6-0ca6-4876-a130-7d4285fd62ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40a502a082624984b003aeb836862bab'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:15.130355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-16e375f6-0ca6-4876-a130-7d4285fd62ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40a502a082624984b003aeb836862bab'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:15.148817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-16e375f6-0ca6-4876-a130-7d4285fd62ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40a502a082624984b003aeb836862bab'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:15.149941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-16e375f6-0ca6-4876-a130-7d4285fd62ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40a502a082624984b003aeb836862bab'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:15.150625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-16e375f6-0ca6-4876-a130-7d4285fd62ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40a502a082624984b003aeb836862bab'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:15.171336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1208/2450] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:40:14 2022] DELETE /identity/v3/users/40a502a082624984b003aeb836862bab => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:15.612531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:15.614146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:15.630269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da82af1-0e97-4de1-b76a-10e1f0ae6349 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:15.691164 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:15.691856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:15.692124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:15.692416 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] PATH_INFO: `/v3/projects/a3bc9a1c1d8f41b5a5eb9055276b9cff` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:15.694119 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3bc9a1c1d8f41b5a5eb9055276b9cff)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:15.695876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:15.805439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3bc9a1c1d8f41b5a5eb9055276b9cff', 'request_id': 'req-dc97f18d-cb53-4661-a2c6-051e01fab6ba'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:15.827865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3bc9a1c1d8f41b5a5eb9055276b9cff', 'request_id': 'req-dc97f18d-cb53-4661-a2c6-051e01fab6ba'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:15.929821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3bc9a1c1d8f41b5a5eb9055276b9cff', 'request_id': 'req-dc97f18d-cb53-4661-a2c6-051e01fab6ba'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:15.963274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] The token cache is being invalidate because project a3bc9a1c1d8f41b5a5eb9055276b9cff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:40:15.964288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:15.965287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dc97f18d-cb53-4661-a2c6-051e01fab6ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:15.967620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1243/2451] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:40:15 2022] DELETE /identity/v3/projects/a3bc9a1c1d8f41b5a5eb9055276b9cff => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:15.973108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f984824-18c6-4912-97cb-c6e62df6b0fa None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:15.973387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f984824-18c6-4912-97cb-c6e62df6b0fa None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:15.973830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f984824-18c6-4912-97cb-c6e62df6b0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:16.013675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1f984824-18c6-4912-97cb-c6e62df6b0fa None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:16.015354 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1f984824-18c6-4912-97cb-c6e62df6b0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:16.163163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1209/2452] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:16.164431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75df803-009e-4d8a-92c3-b22427a2f694 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:16.168390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75df803-009e-4d8a-92c3-b22427a2f694 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:16.168390 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75df803-009e-4d8a-92c3-b22427a2f694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:16.188240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f75df803-009e-4d8a-92c3-b22427a2f694 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:16.189500 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f75df803-009e-4d8a-92c3-b22427a2f694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:16.201572 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1244/2453] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:40:16.206287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f984824-18c6-4912-97cb-c6e62df6b0fa None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:16.220654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f984824-18c6-4912-97cb-c6e62df6b0fa None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:16.232877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c15027-fd9a-4c5a-ad89-510194ad1f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:16.233658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c15027-fd9a-4c5a-ad89-510194ad1f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:16.237493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c15027-fd9a-4c5a-ad89-510194ad1f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:16.237493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c15027-fd9a-4c5a-ad89-510194ad1f4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:16.237493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c15027-fd9a-4c5a-ad89-510194ad1f4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:40:16.237493 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c15027-fd9a-4c5a-ad89-510194ad1f4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:16.239452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c15027-fd9a-4c5a-ad89-510194ad1f4e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:16.261030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1210/2454] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:40:16.266131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75df803-009e-4d8a-92c3-b22427a2f694 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:16.277212 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75df803-009e-4d8a-92c3-b22427a2f694 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:16.282747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc2699-fd95-4840-971b-882b34a2339d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:16.283771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc2699-fd95-4840-971b-882b34a2339d admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:16.284290 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc2699-fd95-4840-971b-882b34a2339d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:16.284773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc2699-fd95-4840-971b-882b34a2339d admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:16.285666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc2699-fd95-4840-971b-882b34a2339d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:16.288440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc2699-fd95-4840-971b-882b34a2339d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:16.432483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1245/2455] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:40:16 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:40:16.435041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c15027-fd9a-4c5a-ad89-510194ad1f4e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:16.453191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c15027-fd9a-4c5a-ad89-510194ad1f4e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:16.528350 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171e87d-0b1c-4db4-b90a-5288556643ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:16.529069 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171e87d-0b1c-4db4-b90a-5288556643ff admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:16.529440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171e87d-0b1c-4db4-b90a-5288556643ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:16.529845 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171e87d-0b1c-4db4-b90a-5288556643ff admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:16.530730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171e87d-0b1c-4db4-b90a-5288556643ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:16.532985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171e87d-0b1c-4db4-b90a-5288556643ff admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:16.542736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:16.700568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1211/2456] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:40:16 2022] POST /identity/v3/users => generated 292 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:40:16.703535 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc2699-fd95-4840-971b-882b34a2339d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:16.722195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc2699-fd95-4840-971b-882b34a2339d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:16.730089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54279886-d70a-4e94-9fc3-3c7d1b5d5440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:16.733854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54279886-d70a-4e94-9fc3-3c7d1b5d5440 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:16.735334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54279886-d70a-4e94-9fc3-3c7d1b5d5440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:16.735334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54279886-d70a-4e94-9fc3-3c7d1b5d5440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:16.735334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54279886-d70a-4e94-9fc3-3c7d1b5d5440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:16.736662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54279886-d70a-4e94-9fc3-3c7d1b5d5440 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:16.761771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1246/2457] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:40:16.761877 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171e87d-0b1c-4db4-b90a-5288556643ff admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:16.784158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171e87d-0b1c-4db4-b90a-5288556643ff admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:16.790742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9580b-5ec9-4419-871c-8ff28cc9ce16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:16.791413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9580b-5ec9-4419-871c-8ff28cc9ce16 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:16.791699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9580b-5ec9-4419-871c-8ff28cc9ce16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:16.791972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9580b-5ec9-4419-871c-8ff28cc9ce16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:16.792740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9580b-5ec9-4419-871c-8ff28cc9ce16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:16.794485 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9580b-5ec9-4419-871c-8ff28cc9ce16 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:16.811161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1212/2458] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:40:16.812623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54279886-d70a-4e94-9fc3-3c7d1b5d5440 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:16.825472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54279886-d70a-4e94-9fc3-3c7d1b5d5440 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:16.830403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79302694-7cc2-4f1e-a618-ca99cf7cc077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:16.831305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79302694-7cc2-4f1e-a618-ca99cf7cc077 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:16.831577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79302694-7cc2-4f1e-a618-ca99cf7cc077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:16.831848 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79302694-7cc2-4f1e-a618-ca99cf7cc077 admin admin] PATH_INFO: `/v3/projects/ead7fd7121224eb9801efd14207e4bc6/users/3d83ba09e1e04173a5f74992d9b631be/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:16.859473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79302694-7cc2-4f1e-a618-ca99cf7cc077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ead7fd7121224eb9801efd14207e4bc6, user_id=3d83ba09e1e04173a5f74992d9b631be, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:16.862367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79302694-7cc2-4f1e-a618-ca99cf7cc077 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:16.973045 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1247/2459] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:40:16 2022] PUT /identity/v3/projects/ead7fd7121224eb9801efd14207e4bc6/users/3d83ba09e1e04173a5f74992d9b631be/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:16.974275 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf717a2-fb9d-4507-91db-9a355b8464a1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:16.974562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf717a2-fb9d-4507-91db-9a355b8464a1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:16.974836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf717a2-fb9d-4507-91db-9a355b8464a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:17.020791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7bf717a2-fb9d-4507-91db-9a355b8464a1 None None] MFA Rules not processed for user `3d83ba09e1e04173a5f74992d9b631be`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:17.022192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7bf717a2-fb9d-4507-91db-9a355b8464a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:17.157160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1213/2460] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:17.161473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79302694-7cc2-4f1e-a618-ca99cf7cc077 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:17.218137 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:17.273837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79302694-7cc2-4f1e-a618-ca99cf7cc077 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:17.278974 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08f1f7-5079-446f-adc5-d9d0a5a71a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:17.279606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08f1f7-5079-446f-adc5-d9d0a5a71a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:17.279890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08f1f7-5079-446f-adc5-d9d0a5a71a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:17.281106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08f1f7-5079-446f-adc5-d9d0a5a71a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:17.281746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08f1f7-5079-446f-adc5-d9d0a5a71a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:17.283768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08f1f7-5079-446f-adc5-d9d0a5a71a46 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:17.398032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1248/2461] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:40:17 2022] POST /identity/v3/projects => generated 383 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:40:17.399424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf717a2-fb9d-4507-91db-9a355b8464a1 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:17.416986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf717a2-fb9d-4507-91db-9a355b8464a1 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:17.530422 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5efeb-9fb8-4d25-9f13-4d5b002d017f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:17.531401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5efeb-9fb8-4d25-9f13-4d5b002d017f admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:17.531690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5efeb-9fb8-4d25-9f13-4d5b002d017f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:17.532048 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5efeb-9fb8-4d25-9f13-4d5b002d017f admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:17.532820 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5efeb-9fb8-4d25-9f13-4d5b002d017f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:17.534787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5efeb-9fb8-4d25-9f13-4d5b002d017f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:17.646481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1214/2462] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:40:17 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:40:17.646679 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08f1f7-5079-446f-adc5-d9d0a5a71a46 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:17.659829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08f1f7-5079-446f-adc5-d9d0a5a71a46 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:17.671467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042cc52-e09d-4dec-9b2d-761dd678158f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:17.675251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042cc52-e09d-4dec-9b2d-761dd678158f admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:17.675251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042cc52-e09d-4dec-9b2d-761dd678158f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:17.675251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042cc52-e09d-4dec-9b2d-761dd678158f admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:17.675251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042cc52-e09d-4dec-9b2d-761dd678158f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:17.675251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042cc52-e09d-4dec-9b2d-761dd678158f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:17.689446 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1249/2463] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:40:17.698269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5efeb-9fb8-4d25-9f13-4d5b002d017f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:17.714401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5efeb-9fb8-4d25-9f13-4d5b002d017f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:17.719701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c1eeb-30db-4e14-895c-bccd8e1daaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:17.720413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c1eeb-30db-4e14-895c-bccd8e1daaa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:17.721036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c1eeb-30db-4e14-895c-bccd8e1daaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:17.726890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c1eeb-30db-4e14-895c-bccd8e1daaa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:17.726890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c1eeb-30db-4e14-895c-bccd8e1daaa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:17.726890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c1eeb-30db-4e14-895c-bccd8e1daaa8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:17.738415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1215/2464] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:40:17.739998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042cc52-e09d-4dec-9b2d-761dd678158f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:17.765425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042cc52-e09d-4dec-9b2d-761dd678158f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:17.778466 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09735e-1499-40f8-8c25-42f086b9ed80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:17.779158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09735e-1499-40f8-8c25-42f086b9ed80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:17.783291 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09735e-1499-40f8-8c25-42f086b9ed80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:17.783596 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09735e-1499-40f8-8c25-42f086b9ed80 admin admin] PATH_INFO: `/v3/projects/6f3d39ce10484767a7ccc88f5bf0d1f8/users/486c27b8306f4a0b8a3b69d3ddd912cf/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:17.804463 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09735e-1499-40f8-8c25-42f086b9ed80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3d39ce10484767a7ccc88f5bf0d1f8, user_id=486c27b8306f4a0b8a3b69d3ddd912cf, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:17.809061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09735e-1499-40f8-8c25-42f086b9ed80 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:17.883509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1250/2465] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:40:17 2022] PUT /identity/v3/projects/6f3d39ce10484767a7ccc88f5bf0d1f8/users/486c27b8306f4a0b8a3b69d3ddd912cf/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:17.894707 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c0046-b9f6-4064-b732-3717abdb157e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:17.895027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c0046-b9f6-4064-b732-3717abdb157e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:17.895334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c0046-b9f6-4064-b732-3717abdb157e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:17.949686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7d0c0046-b9f6-4064-b732-3717abdb157e None None] MFA Rules not processed for user `486c27b8306f4a0b8a3b69d3ddd912cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:17.950915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7d0c0046-b9f6-4064-b732-3717abdb157e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:17.997851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:18.038751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1216/2466] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:40:18.043436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09735e-1499-40f8-8c25-42f086b9ed80 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:18.165166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09735e-1499-40f8-8c25-42f086b9ed80 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:18.179693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94fbb0-dc9a-4de3-a37c-7832afd49f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:18.180674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94fbb0-dc9a-4de3-a37c-7832afd49f46 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:18.181770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94fbb0-dc9a-4de3-a37c-7832afd49f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:18.182056 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94fbb0-dc9a-4de3-a37c-7832afd49f46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:18.182700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94fbb0-dc9a-4de3-a37c-7832afd49f46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:18.185406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94fbb0-dc9a-4de3-a37c-7832afd49f46 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:18.286415 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1251/2467] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:40:18 2022] POST /identity/v3/projects => generated 383 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:40:18.290557 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c0046-b9f6-4064-b732-3717abdb157e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:18.304853 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c0046-b9f6-4064-b732-3717abdb157e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:18.447738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce0da2-4e8b-4f80-bab6-63fb7edc74e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:18.448499 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce0da2-4e8b-4f80-bab6-63fb7edc74e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:18.449582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce0da2-4e8b-4f80-bab6-63fb7edc74e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:18.449934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce0da2-4e8b-4f80-bab6-63fb7edc74e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:18.450796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce0da2-4e8b-4f80-bab6-63fb7edc74e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:18.453143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce0da2-4e8b-4f80-bab6-63fb7edc74e2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:18.466236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753c5d-eb92-4c57-aa70-99f0df964c53 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:18.466834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753c5d-eb92-4c57-aa70-99f0df964c53 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:18.467310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753c5d-eb92-4c57-aa70-99f0df964c53 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:18.470039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1252/2468] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:40:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:40:18.488859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f304d-1f9d-464c-bcfb-faadca25cd52 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:18.489400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f304d-1f9d-464c-bcfb-faadca25cd52 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:18.489846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f304d-1f9d-464c-bcfb-faadca25cd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:18.543711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cb8f304d-1f9d-464c-bcfb-faadca25cd52 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:18.547208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cb8f304d-1f9d-464c-bcfb-faadca25cd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:18.601088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1217/2469] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:40:18 2022] POST /identity/v3/users => generated 298 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:40:18.606730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce0da2-4e8b-4f80-bab6-63fb7edc74e2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:18.632995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce0da2-4e8b-4f80-bab6-63fb7edc74e2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:18.639801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d2375-1854-4c77-babd-9c4e5c9896f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:18.640848 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d2375-1854-4c77-babd-9c4e5c9896f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:18.641346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d2375-1854-4c77-babd-9c4e5c9896f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:18.641927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:18.642168 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d2375-1854-4c77-babd-9c4e5c9896f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:18.644728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d2375-1854-4c77-babd-9c4e5c9896f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:18.647359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d2375-1854-4c77-babd-9c4e5c9896f9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:18.660944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1218/2470] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:40:18.673486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d2375-1854-4c77-babd-9c4e5c9896f9 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:18.689096 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d2375-1854-4c77-babd-9c4e5c9896f9 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:18.689759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1253/2471] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:40:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:18.695165 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837b7ee-1cda-4d99-9ae7-657eac502b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:18.696414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b7ee-1cda-4d99-9ae7-657eac502b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:18.696955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b7ee-1cda-4d99-9ae7-657eac502b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:18.697435 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b7ee-1cda-4d99-9ae7-657eac502b3d admin admin] PATH_INFO: `/v3/projects/ca1363126c0148f7a4918ff7d28f4d06/users/5c0c9466168d46b5a9bc01728135a8be/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:18.725479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837b7ee-1cda-4d99-9ae7-657eac502b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1363126c0148f7a4918ff7d28f4d06, user_id=5c0c9466168d46b5a9bc01728135a8be, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:18.729116 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837b7ee-1cda-4d99-9ae7-657eac502b3d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:18.832775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1219/2472] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:40:18 2022] PUT /identity/v3/projects/ca1363126c0148f7a4918ff7d28f4d06/users/5c0c9466168d46b5a9bc01728135a8be/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:18.837799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f304d-1f9d-464c-bcfb-faadca25cd52 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:18.859150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f304d-1f9d-464c-bcfb-faadca25cd52 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:18.957504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3990e1-5a24-4723-8ccc-525f0f4eed2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:18.958120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3990e1-5a24-4723-8ccc-525f0f4eed2a admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:18.958398 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3990e1-5a24-4723-8ccc-525f0f4eed2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:18.958666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3990e1-5a24-4723-8ccc-525f0f4eed2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:18.959448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3990e1-5a24-4723-8ccc-525f0f4eed2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:18.961324 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3990e1-5a24-4723-8ccc-525f0f4eed2a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:18.986705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1254/2473] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:40:18.988918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837b7ee-1cda-4d99-9ae7-657eac502b3d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:19.010121 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837b7ee-1cda-4d99-9ae7-657eac502b3d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:19.027644 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1118895-3c3e-4c22-9c37-9db42fe4d0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:19.032760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1118895-3c3e-4c22-9c37-9db42fe4d0a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:19.032760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1118895-3c3e-4c22-9c37-9db42fe4d0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:19.032760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1118895-3c3e-4c22-9c37-9db42fe4d0a1 admin admin] PATH_INFO: `/v3/projects/ca1363126c0148f7a4918ff7d28f4d06/users/5c0c9466168d46b5a9bc01728135a8be/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:19.034090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1118895-3c3e-4c22-9c37-9db42fe4d0a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1363126c0148f7a4918ff7d28f4d06, user_id=5c0c9466168d46b5a9bc01728135a8be, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:19.039089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1118895-3c3e-4c22-9c37-9db42fe4d0a1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:19.060083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1220/2474] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:40:18 2022] PUT /identity/v3/projects/ca1363126c0148f7a4918ff7d28f4d06/users/5c0c9466168d46b5a9bc01728135a8be/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:19.072493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e4941-05a3-4391-a381-a5a73d658f45 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:19.072800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e4941-05a3-4391-a381-a5a73d658f45 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:19.073493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e4941-05a3-4391-a381-a5a73d658f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:19.114988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4b0e4941-05a3-4391-a381-a5a73d658f45 None None] MFA Rules not processed for user `5c0c9466168d46b5a9bc01728135a8be`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:19.117473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4b0e4941-05a3-4391-a381-a5a73d658f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:19.286106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1255/2475] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:19.309117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1118895-3c3e-4c22-9c37-9db42fe4d0a1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:19.317160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c1118895-3c3e-4c22-9c37-9db42fe4d0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:19.439146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1118895-3c3e-4c22-9c37-9db42fe4d0a1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:19.454449 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95c423-c9a2-4ae5-a91e-eb8bf76a72b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:19.455132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95c423-c9a2-4ae5-a91e-eb8bf76a72b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:19.455245 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95c423-c9a2-4ae5-a91e-eb8bf76a72b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:19.455476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95c423-c9a2-4ae5-a91e-eb8bf76a72b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:19.493472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:19.522544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95c423-c9a2-4ae5-a91e-eb8bf76a72b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:19.524142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95c423-c9a2-4ae5-a91e-eb8bf76a72b5 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:19.534834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1221/2476] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:40:20.795496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e2c95-3eae-4770-932a-bd2fae1c9975 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:20.796362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e2c95-3eae-4770-932a-bd2fae1c9975 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:20.796641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e2c95-3eae-4770-932a-bd2fae1c9975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:20.829011 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f54e2c95-3eae-4770-932a-bd2fae1c9975 None None] MFA Rules not processed for user `d7f2dbb97068490d924ae132224dc8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:20.830856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f54e2c95-3eae-4770-932a-bd2fae1c9975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:20.901496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:20.905965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1256/2477] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:20.910392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95c423-c9a2-4ae5-a91e-eb8bf76a72b5 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:20.911979 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8d95c423-c9a2-4ae5-a91e-eb8bf76a72b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:21.042574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95c423-c9a2-4ae5-a91e-eb8bf76a72b5 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:21.048804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8af7d-586d-4cb7-824f-ebf024256e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:21.049454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8af7d-586d-4cb7-824f-ebf024256e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:21.049728 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8af7d-586d-4cb7-824f-ebf024256e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:21.049996 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8af7d-586d-4cb7-824f-ebf024256e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:21.050627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8af7d-586d-4cb7-824f-ebf024256e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:21.053434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8af7d-586d-4cb7-824f-ebf024256e0e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:21.159359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1222/2478] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:40:20 2022] POST /identity/v3/projects => generated 373 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:40:21.163671 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e2c95-3eae-4770-932a-bd2fae1c9975 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:21.175197 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e2c95-3eae-4770-932a-bd2fae1c9975 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:21.258052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f208ab-f38c-4b8c-bab8-839af8ff8ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:21.258676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f208ab-f38c-4b8c-bab8-839af8ff8ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:21.258954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f208ab-f38c-4b8c-bab8-839af8ff8ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:21.259496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f208ab-f38c-4b8c-bab8-839af8ff8ce0 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:21.260186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f208ab-f38c-4b8c-bab8-839af8ff8ce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:21.262130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f208ab-f38c-4b8c-bab8-839af8ff8ce0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:21.360453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1257/2479] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:40:21 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:40:21.364561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8af7d-586d-4cb7-824f-ebf024256e0e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:21.375802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8af7d-586d-4cb7-824f-ebf024256e0e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:21.380468 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c9ae5-9cb9-4046-8805-00b01cac3bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:21.381738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c9ae5-9cb9-4046-8805-00b01cac3bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:21.382188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c9ae5-9cb9-4046-8805-00b01cac3bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:21.382466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c9ae5-9cb9-4046-8805-00b01cac3bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:21.383118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c9ae5-9cb9-4046-8805-00b01cac3bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:21.385074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c9ae5-9cb9-4046-8805-00b01cac3bd0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:21.396126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1223/2480] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:40:21.400425 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f208ab-f38c-4b8c-bab8-839af8ff8ce0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:21.411667 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f208ab-f38c-4b8c-bab8-839af8ff8ce0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:21.416588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190a5bb-9f4b-4898-97cf-d61ac7eb81a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:21.417246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190a5bb-9f4b-4898-97cf-d61ac7eb81a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:21.417521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190a5bb-9f4b-4898-97cf-d61ac7eb81a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:21.417825 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190a5bb-9f4b-4898-97cf-d61ac7eb81a9 admin admin] PATH_INFO: `/v3/projects/643f0ddc528d45f1ae5d7a7396b36684/users/a7567b06d54442cca9b337d76aaafd5c/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:21.434408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190a5bb-9f4b-4898-97cf-d61ac7eb81a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643f0ddc528d45f1ae5d7a7396b36684, user_id=a7567b06d54442cca9b337d76aaafd5c, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:21.437261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190a5bb-9f4b-4898-97cf-d61ac7eb81a9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:21.506665 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1258/2481] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:40:21 2022] PUT /identity/v3/projects/643f0ddc528d45f1ae5d7a7396b36684/users/a7567b06d54442cca9b337d76aaafd5c/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:21.511967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c9ae5-9cb9-4046-8805-00b01cac3bd0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:21.522761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c9ae5-9cb9-4046-8805-00b01cac3bd0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:21.580529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dc952-4435-489f-8ee7-9c874b4cb024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:21.581142 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc952-4435-489f-8ee7-9c874b4cb024 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:21.581424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc952-4435-489f-8ee7-9c874b4cb024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:21.581699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc952-4435-489f-8ee7-9c874b4cb024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:21.582379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dc952-4435-489f-8ee7-9c874b4cb024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:21.584157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dc952-4435-489f-8ee7-9c874b4cb024 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:21.595346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1224/2482] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:40:21.598535 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190a5bb-9f4b-4898-97cf-d61ac7eb81a9 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:21.610116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190a5bb-9f4b-4898-97cf-d61ac7eb81a9 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:21.616357 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3169e-c288-4668-931c-b13ac96d09ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:21.623084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3169e-c288-4668-931c-b13ac96d09ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:21.623374 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3169e-c288-4668-931c-b13ac96d09ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:21.625905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3169e-c288-4668-931c-b13ac96d09ba admin admin] PATH_INFO: `/v3/projects/643f0ddc528d45f1ae5d7a7396b36684/users/a7567b06d54442cca9b337d76aaafd5c/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:21.625905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3169e-c288-4668-931c-b13ac96d09ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=643f0ddc528d45f1ae5d7a7396b36684, user_id=a7567b06d54442cca9b337d76aaafd5c, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:21.628424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3169e-c288-4668-931c-b13ac96d09ba admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:21.645296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1259/2483] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:40:21 2022] PUT /identity/v3/projects/643f0ddc528d45f1ae5d7a7396b36684/users/a7567b06d54442cca9b337d76aaafd5c/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:21.759540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dc952-4435-489f-8ee7-9c874b4cb024 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:21.815437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:21.828445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dc952-4435-489f-8ee7-9c874b4cb024 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:21.879288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81773394-e18c-4713-898a-c47f17fdb9bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:21.879944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81773394-e18c-4713-898a-c47f17fdb9bf service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:21.880273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81773394-e18c-4713-898a-c47f17fdb9bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:21.880568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81773394-e18c-4713-898a-c47f17fdb9bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:21.952651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81773394-e18c-4713-898a-c47f17fdb9bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:21.954026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81773394-e18c-4713-898a-c47f17fdb9bf service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:21.963803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1225/2484] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:23.911088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3169e-c288-4668-931c-b13ac96d09ba admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:23.914071 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-69c3169e-c288-4668-931c-b13ac96d09ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:23.930046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3169e-c288-4668-931c-b13ac96d09ba admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:23.934504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258d69d-1be8-44f9-882f-df0efcbddcdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:23.935124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258d69d-1be8-44f9-882f-df0efcbddcdc service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:23.935415 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258d69d-1be8-44f9-882f-df0efcbddcdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:23.935730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258d69d-1be8-44f9-882f-df0efcbddcdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:24.026068 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258d69d-1be8-44f9-882f-df0efcbddcdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:24.027759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258d69d-1be8-44f9-882f-df0efcbddcdc service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:24.054299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1260/2485] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:24.736614 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81773394-e18c-4713-898a-c47f17fdb9bf service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:24.738990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-81773394-e18c-4713-898a-c47f17fdb9bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:24.878035 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81773394-e18c-4713-898a-c47f17fdb9bf service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:24.889373 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:24.890027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:24.890306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:24.890579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] PATH_INFO: `/v3/users/7072bd179b3d4ec2bcc5fcd030c3df59` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:24.891778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7072bd179b3d4ec2bcc5fcd030c3df59)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:24.893944 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:25.188231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7072bd179b3d4ec2bcc5fcd030c3df59'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:25.265617 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7072bd179b3d4ec2bcc5fcd030c3df59'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:25.268283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:25.287750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7072bd179b3d4ec2bcc5fcd030c3df59'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:25.288783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7072bd179b3d4ec2bcc5fcd030c3df59'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:25.289706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7072bd179b3d4ec2bcc5fcd030c3df59'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:25.308739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1226/2486] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:40:24 2022] DELETE /identity/v3/users/7072bd179b3d4ec2bcc5fcd030c3df59 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:25.313141 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf929a-0578-430b-9f32-a373d73eb5bf None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:25.313615 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf929a-0578-430b-9f32-a373d73eb5bf None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:25.313694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf929a-0578-430b-9f32-a373d73eb5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:25.374530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-92cf929a-0578-430b-9f32-a373d73eb5bf None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:25.376735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-92cf929a-0578-430b-9f32-a373d73eb5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:25.489185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:25.536305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1261/2487] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:25.547961 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:25.549482 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:25.694913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec1953-cbdd-4b7d-9f58-9ba07b9926d3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:25.699893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf570e4c-f717-40a9-8d05-4c02c2fb1b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:25.701188 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf570e4c-f717-40a9-8d05-4c02c2fb1b1d service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:25.701666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf570e4c-f717-40a9-8d05-4c02c2fb1b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:25.714007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf570e4c-f717-40a9-8d05-4c02c2fb1b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:25.717068 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf570e4c-f717-40a9-8d05-4c02c2fb1b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:25.719788 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf570e4c-f717-40a9-8d05-4c02c2fb1b1d service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:25.735260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1227/2488] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:26.353030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf929a-0578-430b-9f32-a373d73eb5bf None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:26.354607 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-92cf929a-0578-430b-9f32-a373d73eb5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:26.370311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf929a-0578-430b-9f32-a373d73eb5bf None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:26.375692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:26.376361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:26.376645 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:26.376919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] PATH_INFO: `/v3/projects/7a32cf48d21c4ec9b39b3f5fa2ee752a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:26.379130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a32cf48d21c4ec9b39b3f5fa2ee752a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:26.384782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:26.510246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a32cf48d21c4ec9b39b3f5fa2ee752a', 'request_id': 'req-dd3183e9-eac2-4639-815b-6c983b2d6aa0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:26.520040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a32cf48d21c4ec9b39b3f5fa2ee752a', 'request_id': 'req-dd3183e9-eac2-4639-815b-6c983b2d6aa0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:26.626041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a32cf48d21c4ec9b39b3f5fa2ee752a', 'request_id': 'req-dd3183e9-eac2-4639-815b-6c983b2d6aa0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:26.672989 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] The token cache is being invalidate because project 7a32cf48d21c4ec9b39b3f5fa2ee752a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:40:26.673511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:26.674489 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dd3183e9-eac2-4639-815b-6c983b2d6aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:26.677487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1262/2489] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:40:26 2022] DELETE /identity/v3/projects/7a32cf48d21c4ec9b39b3f5fa2ee752a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:26.688051 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b22ee-8da0-482a-b276-17f4812a37f0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:26.688641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b22ee-8da0-482a-b276-17f4812a37f0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:26.688641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b22ee-8da0-482a-b276-17f4812a37f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:26.718232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f53b22ee-8da0-482a-b276-17f4812a37f0 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:26.719504 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f53b22ee-8da0-482a-b276-17f4812a37f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:26.842580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1228/2490] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:26.844605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c50fa3-a5a1-4a7c-8f7b-574f1a813220 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:26.844899 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c50fa3-a5a1-4a7c-8f7b-574f1a813220 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:26.845288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c50fa3-a5a1-4a7c-8f7b-574f1a813220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:26.870169 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-06c50fa3-a5a1-4a7c-8f7b-574f1a813220 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:26.871549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-06c50fa3-a5a1-4a7c-8f7b-574f1a813220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:26.879905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1263/2491] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:26.885467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b22ee-8da0-482a-b276-17f4812a37f0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:26.895372 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b22ee-8da0-482a-b276-17f4812a37f0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:26.904033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d6c34-8c07-4958-a500-702ff3d5d296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:26.904658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d6c34-8c07-4958-a500-702ff3d5d296 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:26.906459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d6c34-8c07-4958-a500-702ff3d5d296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:26.906459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d6c34-8c07-4958-a500-702ff3d5d296 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:26.906459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d6c34-8c07-4958-a500-702ff3d5d296 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:40:26.906459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d6c34-8c07-4958-a500-702ff3d5d296 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:26.908169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d6c34-8c07-4958-a500-702ff3d5d296 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:26.921424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1229/2492] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:40:26.926708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c50fa3-a5a1-4a7c-8f7b-574f1a813220 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:26.937166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c50fa3-a5a1-4a7c-8f7b-574f1a813220 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:26.944611 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3562a-c351-4133-a5c8-16b784fe7996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:26.945238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3562a-c351-4133-a5c8-16b784fe7996 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:26.945516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3562a-c351-4133-a5c8-16b784fe7996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:26.945787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3562a-c351-4133-a5c8-16b784fe7996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:26.946435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3562a-c351-4133-a5c8-16b784fe7996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:26.948409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3562a-c351-4133-a5c8-16b784fe7996 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:27.068340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1264/2493] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:40:26 2022] POST /identity/v3/projects => generated 369 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:40:27.072870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d6c34-8c07-4958-a500-702ff3d5d296 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:27.086181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d6c34-8c07-4958-a500-702ff3d5d296 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:27.157526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1276d-7881-49dc-8fdf-77e209b22fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:27.158173 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1276d-7881-49dc-8fdf-77e209b22fbb admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:27.158461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1276d-7881-49dc-8fdf-77e209b22fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:27.158738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1276d-7881-49dc-8fdf-77e209b22fbb admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:27.159380 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1276d-7881-49dc-8fdf-77e209b22fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:27.161353 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1276d-7881-49dc-8fdf-77e209b22fbb admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:27.172423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:27.281169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1230/2494] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:40:27 2022] POST /identity/v3/users => generated 285 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:40:27.282269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3562a-c351-4133-a5c8-16b784fe7996 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:27.310503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3562a-c351-4133-a5c8-16b784fe7996 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:27.322090 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46465e-4838-4dd0-9683-0c749fc29cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:27.322726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46465e-4838-4dd0-9683-0c749fc29cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:27.323180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46465e-4838-4dd0-9683-0c749fc29cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:27.323460 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46465e-4838-4dd0-9683-0c749fc29cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:27.324154 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46465e-4838-4dd0-9683-0c749fc29cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:27.326192 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46465e-4838-4dd0-9683-0c749fc29cd7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:27.348200 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1265/2495] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:40:27.350664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1276d-7881-49dc-8fdf-77e209b22fbb admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:27.365931 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1276d-7881-49dc-8fdf-77e209b22fbb admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:27.379470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483fa31-45f9-4165-b5a5-49e517f2e79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:27.380317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483fa31-45f9-4165-b5a5-49e517f2e79f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:27.380820 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483fa31-45f9-4165-b5a5-49e517f2e79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:27.381490 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483fa31-45f9-4165-b5a5-49e517f2e79f admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:27.382335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483fa31-45f9-4165-b5a5-49e517f2e79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:27.385017 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483fa31-45f9-4165-b5a5-49e517f2e79f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:27.404580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1231/2496] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:40:27.406353 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46465e-4838-4dd0-9683-0c749fc29cd7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:27.427411 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46465e-4838-4dd0-9683-0c749fc29cd7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:27.453214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503e289-7ca9-4b34-9ddc-be9a841b7169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:27.454118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503e289-7ca9-4b34-9ddc-be9a841b7169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:27.454507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503e289-7ca9-4b34-9ddc-be9a841b7169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:27.454887 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503e289-7ca9-4b34-9ddc-be9a841b7169 admin admin] PATH_INFO: `/v3/projects/15057d21eacf42daa877880270e083c7/users/ff130beddc3c4466a2830d22dedf5c15/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:27.507555 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503e289-7ca9-4b34-9ddc-be9a841b7169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15057d21eacf42daa877880270e083c7, user_id=ff130beddc3c4466a2830d22dedf5c15, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:27.515225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503e289-7ca9-4b34-9ddc-be9a841b7169 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:27.596338 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1266/2497] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:40:27 2022] PUT /identity/v3/projects/15057d21eacf42daa877880270e083c7/users/ff130beddc3c4466a2830d22dedf5c15/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:27.604491 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800ccbb-b7ff-4400-8a7c-af0b70b01bbf None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:27.604801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800ccbb-b7ff-4400-8a7c-af0b70b01bbf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:27.605089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800ccbb-b7ff-4400-8a7c-af0b70b01bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:27.645988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0800ccbb-b7ff-4400-8a7c-af0b70b01bbf None None] MFA Rules not processed for user `ff130beddc3c4466a2830d22dedf5c15`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:27.648322 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0800ccbb-b7ff-4400-8a7c-af0b70b01bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:27.738063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1232/2498] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:27.942536 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503e289-7ca9-4b34-9ddc-be9a841b7169 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:27.945742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6503e289-7ca9-4b34-9ddc-be9a841b7169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:28.068941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503e289-7ca9-4b34-9ddc-be9a841b7169 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:28.082402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567c00a-089a-4bda-b9ea-caf39d7ce22d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb2129c006f4ddeba7d128a8d43d5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:28.083030 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567c00a-089a-4bda-b9ea-caf39d7ce22d service glance] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:28.089811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567c00a-089a-4bda-b9ea-caf39d7ce22d service glance] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:28.089811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567c00a-089a-4bda-b9ea-caf39d7ce22d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:28.091313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0567c00a-089a-4bda-b9ea-caf39d7ce22d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:28.143862 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:28.233660 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567c00a-089a-4bda-b9ea-caf39d7ce22d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:28.235172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567c00a-089a-4bda-b9ea-caf39d7ce22d service glance] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:28.265066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1267/2499] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:28.286517 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800ccbb-b7ff-4400-8a7c-af0b70b01bbf None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:28.305167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800ccbb-b7ff-4400-8a7c-af0b70b01bbf None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:28.311885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50e039-ee6a-487b-9501-2392246f4ae3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb2129c006f4ddeba7d128a8d43d5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:28.313686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50e039-ee6a-487b-9501-2392246f4ae3 service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:28.313686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50e039-ee6a-487b-9501-2392246f4ae3 service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:28.313686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50e039-ee6a-487b-9501-2392246f4ae3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:28.331243 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50e039-ee6a-487b-9501-2392246f4ae3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:28.332884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50e039-ee6a-487b-9501-2392246f4ae3 service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:28.359298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1233/2500] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:28.684207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caf973-abf4-437e-a04f-db7c2adabb6e None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:28.684719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caf973-abf4-437e-a04f-db7c2adabb6e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:28.684814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caf973-abf4-437e-a04f-db7c2adabb6e None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:28.686293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1268/2501] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:40:28.692500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c8df9-8375-4d4c-8db7-7eaad473533c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:28.692981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c8df9-8375-4d4c-8db7-7eaad473533c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:28.693506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c8df9-8375-4d4c-8db7-7eaad473533c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:28.728419 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-673c8df9-8375-4d4c-8db7-7eaad473533c None None] MFA Rules not processed for user `baffd4f6a8ad4a6e91d8d9c61cc1dce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:28.730450 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-673c8df9-8375-4d4c-8db7-7eaad473533c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:28.793572 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570a5a8-ba21-45e5-a884-7519e52e8eed None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:28.794084 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570a5a8-ba21-45e5-a884-7519e52e8eed None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:28.794530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570a5a8-ba21-45e5-a884-7519e52e8eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:28.823056 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1234/2502] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:40:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:40:28.830604 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c8df9-8375-4d4c-8db7-7eaad473533c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:28.831551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5570a5a8-ba21-45e5-a884-7519e52e8eed None None] MFA Rules not processed for user `a7567b06d54442cca9b337d76aaafd5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:28.832880 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5570a5a8-ba21-45e5-a884-7519e52e8eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:28.833086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-673c8df9-8375-4d4c-8db7-7eaad473533c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:28.851768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c8df9-8375-4d4c-8db7-7eaad473533c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:28.860205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af962a5-e304-4b24-be63-23a0383687be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:28.860933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af962a5-e304-4b24-be63-23a0383687be service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:28.861701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af962a5-e304-4b24-be63-23a0383687be service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:28.862104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af962a5-e304-4b24-be63-23a0383687be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:28.872875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af962a5-e304-4b24-be63-23a0383687be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:28.875209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af962a5-e304-4b24-be63-23a0383687be service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:28.891453 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1235/2503] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:28.923317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1269/2504] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:28.959974 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af962a5-e304-4b24-be63-23a0383687be service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:28.959974 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6af962a5-e304-4b24-be63-23a0383687be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:28.964031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af962a5-e304-4b24-be63-23a0383687be service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:28.971321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddeb9d0-a791-4552-9de3-fd2735cba6d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:28.972153 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddeb9d0-a791-4552-9de3-fd2735cba6d5 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:28.979541 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddeb9d0-a791-4552-9de3-fd2735cba6d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:28.980295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddeb9d0-a791-4552-9de3-fd2735cba6d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:28.985046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8ddeb9d0-a791-4552-9de3-fd2735cba6d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:29.009498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52c7d-8151-47ed-b118-6e0b13b5d288 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:29.009498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52c7d-8151-47ed-b118-6e0b13b5d288 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:29.009498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52c7d-8151-47ed-b118-6e0b13b5d288 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:29.009498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1270/2505] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 22:40:29.022450 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76837952-890c-482c-baaa-2e582a52c729 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:29.022866 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76837952-890c-482c-baaa-2e582a52c729 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:29.023853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76837952-890c-482c-baaa-2e582a52c729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:29.050467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-76837952-890c-482c-baaa-2e582a52c729 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:29.052068 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-76837952-890c-482c-baaa-2e582a52c729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:29.080348 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1271/2506] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:40:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:29.091592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:29.104415 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76837952-890c-482c-baaa-2e582a52c729 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:29.107755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-76837952-890c-482c-baaa-2e582a52c729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:29.108941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddeb9d0-a791-4552-9de3-fd2735cba6d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:29.110550 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddeb9d0-a791-4552-9de3-fd2735cba6d5 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:29.124302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76837952-890c-482c-baaa-2e582a52c729 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:29.128714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1236/2507] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:29.130393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedee03a-832f-44eb-be32-b29d17faf921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:29.131266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedee03a-832f-44eb-be32-b29d17faf921 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:29.131554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedee03a-832f-44eb-be32-b29d17faf921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:29.131826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedee03a-832f-44eb-be32-b29d17faf921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:29.152549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedee03a-832f-44eb-be32-b29d17faf921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:29.154149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedee03a-832f-44eb-be32-b29d17faf921 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:29.179334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1272/2508] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:29.632892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddeb9d0-a791-4552-9de3-fd2735cba6d5 service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:29.634885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8ddeb9d0-a791-4552-9de3-fd2735cba6d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:29.779547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddeb9d0-a791-4552-9de3-fd2735cba6d5 service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:29.790580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:29.791447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:29.792881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:29.794007 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] PATH_INFO: `/v3/users/3d83ba09e1e04173a5f74992d9b631be` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:29.796527 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d83ba09e1e04173a5f74992d9b631be)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:29.799331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:29.989478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d83ba09e1e04173a5f74992d9b631be'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:30.054203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d83ba09e1e04173a5f74992d9b631be'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:30.067591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d83ba09e1e04173a5f74992d9b631be'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:30.068835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d83ba09e1e04173a5f74992d9b631be'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:30.070428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d83ba09e1e04173a5f74992d9b631be'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:30.083845 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1237/2509] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:40:29 2022] DELETE /identity/v3/users/3d83ba09e1e04173a5f74992d9b631be => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:30.089113 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb896-d240-4f35-b5b0-388ae0885b6f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:30.089624 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb896-d240-4f35-b5b0-388ae0885b6f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:30.089758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb896-d240-4f35-b5b0-388ae0885b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:30.118187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-835cb896-d240-4f35-b5b0-388ae0885b6f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:30.118187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-835cb896-d240-4f35-b5b0-388ae0885b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:30.180473 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1273/2510] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:30.190884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:30.192748 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:30.265238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1cd41-5044-4e1e-9364-bd0be15aba20 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:30.269387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ee14d-9ab6-4bde-8767-cba663877434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:30.269885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ee14d-9ab6-4bde-8767-cba663877434 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:30.270172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ee14d-9ab6-4bde-8767-cba663877434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:30.270446 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ee14d-9ab6-4bde-8767-cba663877434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:30.285723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ee14d-9ab6-4bde-8767-cba663877434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:30.290664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ee14d-9ab6-4bde-8767-cba663877434 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:30.301639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1238/2511] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:30.748891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cb896-d240-4f35-b5b0-388ae0885b6f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:30.750581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-835cb896-d240-4f35-b5b0-388ae0885b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:30.766391 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cb896-d240-4f35-b5b0-388ae0885b6f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:30.771685 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243501-9219-4989-ae10-a303105c287e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:30.778220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243501-9219-4989-ae10-a303105c287e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:30.778220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243501-9219-4989-ae10-a303105c287e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:30.778220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243501-9219-4989-ae10-a303105c287e admin admin] PATH_INFO: `/v3/projects/ead7fd7121224eb9801efd14207e4bc6` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:30.778220 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63243501-9219-4989-ae10-a303105c287e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ead7fd7121224eb9801efd14207e4bc6)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:30.778812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63243501-9219-4989-ae10-a303105c287e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:30.788676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:30.922339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63243501-9219-4989-ae10-a303105c287e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ead7fd7121224eb9801efd14207e4bc6', 'request_id': 'req-63243501-9219-4989-ae10-a303105c287e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:30.941242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63243501-9219-4989-ae10-a303105c287e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ead7fd7121224eb9801efd14207e4bc6', 'request_id': 'req-63243501-9219-4989-ae10-a303105c287e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:31.060897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63243501-9219-4989-ae10-a303105c287e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ead7fd7121224eb9801efd14207e4bc6', 'request_id': 'req-63243501-9219-4989-ae10-a303105c287e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:31.081927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63243501-9219-4989-ae10-a303105c287e admin admin] The token cache is being invalidate because project ead7fd7121224eb9801efd14207e4bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:40:31.082462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63243501-9219-4989-ae10-a303105c287e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:31.083229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63243501-9219-4989-ae10-a303105c287e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:31.085673 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1274/2512] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:40:30 2022] DELETE /identity/v3/projects/ead7fd7121224eb9801efd14207e4bc6 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:31.090453 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ee14d-9ab6-4bde-8767-cba663877434 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:31.102132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ee14d-9ab6-4bde-8767-cba663877434 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:31.112097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:31.163795 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:31.164462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:31.164739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:31.165011 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] PATH_INFO: `/v3/users/486c27b8306f4a0b8a3b69d3ddd912cf` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:31.166184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] RBAC: Authorizing `identity:delete_user(user_id=486c27b8306f4a0b8a3b69d3ddd912cf)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:31.168291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:31.367022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '486c27b8306f4a0b8a3b69d3ddd912cf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:31.436849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '486c27b8306f4a0b8a3b69d3ddd912cf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:31.450031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '486c27b8306f4a0b8a3b69d3ddd912cf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:31.450804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '486c27b8306f4a0b8a3b69d3ddd912cf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:31.451470 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '486c27b8306f4a0b8a3b69d3ddd912cf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:31.464687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1239/2513] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:40:31 2022] DELETE /identity/v3/users/486c27b8306f4a0b8a3b69d3ddd912cf => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:31.710964 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243501-9219-4989-ae10-a303105c287e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:31.712463 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-63243501-9219-4989-ae10-a303105c287e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:31.791230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243501-9219-4989-ae10-a303105c287e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:31.871883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc3d3b-2f9d-4fdf-9c80-e50a42ef6cea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:31.873112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc3d3b-2f9d-4fdf-9c80-e50a42ef6cea service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:31.873648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc3d3b-2f9d-4fdf-9c80-e50a42ef6cea service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:31.874114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc3d3b-2f9d-4fdf-9c80-e50a42ef6cea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:31.877243 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-04fc3d3b-2f9d-4fdf-9c80-e50a42ef6cea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:31.898617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc3d3b-2f9d-4fdf-9c80-e50a42ef6cea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:31.901131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc3d3b-2f9d-4fdf-9c80-e50a42ef6cea service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:31.915795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1275/2514] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:40:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:40:31.994830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:31.996231 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:32.141051 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439a140-2d9a-4845-81da-0bd1786dc2df admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:32.154906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:32.155629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:32.156089 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:32.156368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] PATH_INFO: `/v3/projects/6f3d39ce10484767a7ccc88f5bf0d1f8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:32.164132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f3d39ce10484767a7ccc88f5bf0d1f8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:32.166095 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:32.298368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:32.310564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f3d39ce10484767a7ccc88f5bf0d1f8', 'request_id': 'req-bdc847f7-31fe-42bf-8977-44f467ef9492'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:32.317316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f3d39ce10484767a7ccc88f5bf0d1f8', 'request_id': 'req-bdc847f7-31fe-42bf-8977-44f467ef9492'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:32.386157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f3d39ce10484767a7ccc88f5bf0d1f8', 'request_id': 'req-bdc847f7-31fe-42bf-8977-44f467ef9492'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:32.398990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] The token cache is being invalidate because project 6f3d39ce10484767a7ccc88f5bf0d1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:40:32.399278 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:32.399933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-bdc847f7-31fe-42bf-8977-44f467ef9492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:32.405542 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1240/2515] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:40:31 2022] DELETE /identity/v3/projects/6f3d39ce10484767a7ccc88f5bf0d1f8 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:32.405800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc3d3b-2f9d-4fdf-9c80-e50a42ef6cea service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:32.417442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc3d3b-2f9d-4fdf-9c80-e50a42ef6cea service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:32.453823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:32.486289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae8fec-4b88-46d7-beaa-9303983cfaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:32.486922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae8fec-4b88-46d7-beaa-9303983cfaf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:32.487201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae8fec-4b88-46d7-beaa-9303983cfaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:32.487476 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae8fec-4b88-46d7-beaa-9303983cfaf9 admin admin] PATH_INFO: `/v3/users/5c0c9466168d46b5a9bc01728135a8be` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:32.488623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae8fec-4b88-46d7-beaa-9303983cfaf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c0c9466168d46b5a9bc01728135a8be)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:32.490614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae8fec-4b88-46d7-beaa-9303983cfaf9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:32.538533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66eb4-ec0f-4b7d-87e6-238002b56b92 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:32.538819 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66eb4-ec0f-4b7d-87e6-238002b56b92 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:32.539095 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66eb4-ec0f-4b7d-87e6-238002b56b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:32.558871 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c0a66eb4-ec0f-4b7d-87e6-238002b56b92 None None] MFA Rules not processed for user `d7f2dbb97068490d924ae132224dc8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:32.560146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c0a66eb4-ec0f-4b7d-87e6-238002b56b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:32.623113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1241/2516] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:32.720511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6eae8fec-4b88-46d7-beaa-9303983cfaf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c0c9466168d46b5a9bc01728135a8be'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:32.791093 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6eae8fec-4b88-46d7-beaa-9303983cfaf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c0c9466168d46b5a9bc01728135a8be'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:32.810274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6eae8fec-4b88-46d7-beaa-9303983cfaf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c0c9466168d46b5a9bc01728135a8be'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:32.822734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6eae8fec-4b88-46d7-beaa-9303983cfaf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c0c9466168d46b5a9bc01728135a8be'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:32.824036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6eae8fec-4b88-46d7-beaa-9303983cfaf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c0c9466168d46b5a9bc01728135a8be'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:32.835711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1276/2517] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:40:32 2022] DELETE /identity/v3/users/5c0c9466168d46b5a9bc01728135a8be => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:33.604739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a66eb4-ec0f-4b7d-87e6-238002b56b92 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:33.607166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c0a66eb4-ec0f-4b7d-87e6-238002b56b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:33.630397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a66eb4-ec0f-4b7d-87e6-238002b56b92 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:33.719870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:33.721004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:33.721475 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:33.721906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] PATH_INFO: `/v3/projects/ca1363126c0148f7a4918ff7d28f4d06` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:33.723397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1363126c0148f7a4918ff7d28f4d06)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:33.726812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:33.891332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1363126c0148f7a4918ff7d28f4d06', 'request_id': 'req-1770e97f-dace-4058-93b7-fda1061cb985'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:33.898661 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1363126c0148f7a4918ff7d28f4d06', 'request_id': 'req-1770e97f-dace-4058-93b7-fda1061cb985'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:33.979376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1363126c0148f7a4918ff7d28f4d06', 'request_id': 'req-1770e97f-dace-4058-93b7-fda1061cb985'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:33.986652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:33.992163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] The token cache is being invalidate because project ca1363126c0148f7a4918ff7d28f4d06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:40:33.992514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:33.993164 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:40:34.001506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1242/2518] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:40:33 2022] DELETE /identity/v3/projects/ca1363126c0148f7a4918ff7d28f4d06 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:40:34.067747 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d259f9-3161-4cee-a0ea-d48a3642e632 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:34.068982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d259f9-3161-4cee-a0ea-d48a3642e632 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:34.069285 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d259f9-3161-4cee-a0ea-d48a3642e632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:34.094491 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-71d259f9-3161-4cee-a0ea-d48a3642e632 None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:34.096410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-71d259f9-3161-4cee-a0ea-d48a3642e632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:34.176595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1277/2519] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:40:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:34.189958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:40:34.191385 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:34.268041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770e97f-dace-4058-93b7-fda1061cb985 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:40:34.274325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427336f-dd83-489b-82bc-8c9fca35c1c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:40:34.274937 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427336f-dd83-489b-82bc-8c9fca35c1c8 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:34.275218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427336f-dd83-489b-82bc-8c9fca35c1c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:34.275519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427336f-dd83-489b-82bc-8c9fca35c1c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:34.284308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427336f-dd83-489b-82bc-8c9fca35c1c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:40:34.285882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427336f-dd83-489b-82bc-8c9fca35c1c8 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:40:34.294863 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1243/2520] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:40:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:40:37.600459 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25deb2-2f31-465b-9b54-9d46d25e6a15 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:40:37.601423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25deb2-2f31-465b-9b54-9d46d25e6a15 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:40:37.601701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25deb2-2f31-465b-9b54-9d46d25e6a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:40:37.620725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1f25deb2-2f31-465b-9b54-9d46d25e6a15 None None] MFA Rules not processed for user `394c67bcc3864bb2aca2f4faec834029`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:40:37.621973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1f25deb2-2f31-465b-9b54-9d46d25e6a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:40:37.692705 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:37.703232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1278/2521] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:40:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:40:44.390459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=33 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:40:47.767347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:41:04.910898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d089e09-07d4-4339-a881-429104c6ddff None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:41:04.912075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d089e09-07d4-4339-a881-429104c6ddff None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:41:04.912251 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d089e09-07d4-4339-a881-429104c6ddff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:41:04.931226 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6d089e09-07d4-4339-a881-429104c6ddff None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:41:04.932480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6d089e09-07d4-4339-a881-429104c6ddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:41:04.940083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1244/2522] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 22:41:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:41:04.963481 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25deb2-2f31-465b-9b54-9d46d25e6a15 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:41:04.978367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25deb2-2f31-465b-9b54-9d46d25e6a15 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:41:04.983511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e541e46-b514-4024-86ad-66f5df1f79b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:41:04.984137 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e541e46-b514-4024-86ad-66f5df1f79b7 service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:41:04.984414 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e541e46-b514-4024-86ad-66f5df1f79b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:41:04.984682 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e541e46-b514-4024-86ad-66f5df1f79b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:41:04.994889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e541e46-b514-4024-86ad-66f5df1f79b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:41:04.996782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e541e46-b514-4024-86ad-66f5df1f79b7 service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:41:05.009106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1279/2523] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:41:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:41:14.926336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:41:15.005201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:41:20.644609 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d089e09-07d4-4339-a881-429104c6ddff None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:41:20.646696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6d089e09-07d4-4339-a881-429104c6ddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:41:20.722335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d089e09-07d4-4339-a881-429104c6ddff None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:41:20.727714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:41:20.728371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:41:20.728654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:41:20.728934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] PATH_INFO: `/v3/users/ff130beddc3c4466a2830d22dedf5c15` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:41:20.730097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff130beddc3c4466a2830d22dedf5c15)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:41:20.732040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:41:20.905118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff130beddc3c4466a2830d22dedf5c15'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:41:20.960637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff130beddc3c4466a2830d22dedf5c15'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:41:20.973420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff130beddc3c4466a2830d22dedf5c15'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:41:20.974070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff130beddc3c4466a2830d22dedf5c15'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:41:20.975177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff130beddc3c4466a2830d22dedf5c15'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:41:20.989131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1245/2524] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:41:20 2022] DELETE /identity/v3/users/ff130beddc3c4466a2830d22dedf5c15 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:41:20.994457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c1e0a-7309-4663-ad32-b347a4e5dc53 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:41:20.998760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c1e0a-7309-4663-ad32-b347a4e5dc53 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:41:20.998760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c1e0a-7309-4663-ad32-b347a4e5dc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:41:21.015224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bd6c1e0a-7309-4663-ad32-b347a4e5dc53 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:41:21.016479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bd6c1e0a-7309-4663-ad32-b347a4e5dc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:41:21.106626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1280/2525] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:41:21.116430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:41:21.117851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:41:21.175445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd473b79-f2d2-4ddc-8944-f4fb0b4759ba admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:41:21.180182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979d00d-6fc9-4dce-9d3f-ccb5e00037d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:41:21.180777 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979d00d-6fc9-4dce-9d3f-ccb5e00037d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:41:21.181054 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979d00d-6fc9-4dce-9d3f-ccb5e00037d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:41:21.181325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979d00d-6fc9-4dce-9d3f-ccb5e00037d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:41:21.190413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979d00d-6fc9-4dce-9d3f-ccb5e00037d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:41:21.191828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979d00d-6fc9-4dce-9d3f-ccb5e00037d0 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:41:21.202022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1246/2526] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:41:21.597313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c1e0a-7309-4663-ad32-b347a4e5dc53 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:41:21.598756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bd6c1e0a-7309-4663-ad32-b347a4e5dc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:41:21.612661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c1e0a-7309-4663-ad32-b347a4e5dc53 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:41:21.622028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:41:21.622725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:41:21.623071 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:41:21.623279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] PATH_INFO: `/v3/projects/15057d21eacf42daa877880270e083c7` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:41:21.624478 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15057d21eacf42daa877880270e083c7)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:41:21.626397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:41:21.712920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15057d21eacf42daa877880270e083c7', 'request_id': 'req-c65f74ec-29a2-4d6c-a413-505434ed0ab1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:41:21.719633 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15057d21eacf42daa877880270e083c7', 'request_id': 'req-c65f74ec-29a2-4d6c-a413-505434ed0ab1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:41:21.773273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15057d21eacf42daa877880270e083c7', 'request_id': 'req-c65f74ec-29a2-4d6c-a413-505434ed0ab1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:41:21.785321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] The token cache is being invalidate because project 15057d21eacf42daa877880270e083c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:41:21.785616 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:41:21.786484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c65f74ec-29a2-4d6c-a413-505434ed0ab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:41:21.789086 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1281/2527] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:41:21 2022] DELETE /identity/v3/projects/15057d21eacf42daa877880270e083c7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:41:23.618409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979d00d-6fc9-4dce-9d3f-ccb5e00037d0 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:41:23.627365 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979d00d-6fc9-4dce-9d3f-ccb5e00037d0 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:41:23.648181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:41:23.680812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e67c5-7ab7-4fd9-94f8-9de4e98a545b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:41:23.681408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e67c5-7ab7-4fd9-94f8-9de4e98a545b service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:41:23.681682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e67c5-7ab7-4fd9-94f8-9de4e98a545b service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:41:23.684169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e67c5-7ab7-4fd9-94f8-9de4e98a545b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:41:23.684169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-374e67c5-7ab7-4fd9-94f8-9de4e98a545b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:41:23.746325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e67c5-7ab7-4fd9-94f8-9de4e98a545b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:41:23.747761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e67c5-7ab7-4fd9-94f8-9de4e98a545b service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:41:23.760130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1247/2528] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:41:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:41:31.782023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:41:33.761555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=40 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:41:35.097627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285af36-6eb0-4ae8-8dde-8a615d57ec2e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:41:35.098184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285af36-6eb0-4ae8-8dde-8a615d57ec2e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:41:35.098184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285af36-6eb0-4ae8-8dde-8a615d57ec2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:41:35.118308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3285af36-6eb0-4ae8-8dde-8a615d57ec2e None None] MFA Rules not processed for user `def1da13ff2a47ada8e5f453061e3f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:41:35.119642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3285af36-6eb0-4ae8-8dde-8a615d57ec2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:41:35.178587 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1282/2529] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:41:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:41:38.245383 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e67c5-7ab7-4fd9-94f8-9de4e98a545b service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:41:38.246924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-374e67c5-7ab7-4fd9-94f8-9de4e98a545b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:41:38.258108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e67c5-7ab7-4fd9-94f8-9de4e98a545b service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:41:38.262046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ad8f7-2bda-4652-a186-261756acfc19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:41:38.262647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ad8f7-2bda-4652-a186-261756acfc19 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:41:38.262930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ad8f7-2bda-4652-a186-261756acfc19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:41:38.263206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ad8f7-2bda-4652-a186-261756acfc19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:41:38.264926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-db8ad8f7-2bda-4652-a186-261756acfc19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:41:38.335355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ad8f7-2bda-4652-a186-261756acfc19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:41:38.336780 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ad8f7-2bda-4652-a186-261756acfc19 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:41:38.347475 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1248/2530] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:41:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:41:43.635026 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271c1e4-e813-487d-9139-15b87c5b35d4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:41:43.635448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271c1e4-e813-487d-9139-15b87c5b35d4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:41:43.635577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271c1e4-e813-487d-9139-15b87c5b35d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:41:43.653917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9271c1e4-e813-487d-9139-15b87c5b35d4 None None] MFA Rules not processed for user `966045fbe69f43b4b1a825da3ad571b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:41:43.655112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9271c1e4-e813-487d-9139-15b87c5b35d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:41:43.709317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1283/2531] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:41:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:41:48.343144 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=30 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:41:53.702573 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=50 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:42:12.791086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea11e7-0f6a-478c-bd5c-fbf5a4f9dc78 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:12.791569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea11e7-0f6a-478c-bd5c-fbf5a4f9dc78 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:12.791681 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea11e7-0f6a-478c-bd5c-fbf5a4f9dc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:12.811164 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c5ea11e7-0f6a-478c-bd5c-fbf5a4f9dc78 None None] MFA Rules not processed for user `05ef5ec6afb84c4abc0cb4862f041dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:42:12.812424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c5ea11e7-0f6a-478c-bd5c-fbf5a4f9dc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:12.867574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1249/2532] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:42:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:42:19.257230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271c1e4-e813-487d-9139-15b87c5b35d4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:19.267419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271c1e4-e813-487d-9139-15b87c5b35d4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:19.271804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea6bc0-badb-49c1-b597-e19ab30abbb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:19.272455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6bc0-badb-49c1-b597-e19ab30abbb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:19.272740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6bc0-badb-49c1-b597-e19ab30abbb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:19.273027 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6bc0-badb-49c1-b597-e19ab30abbb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:19.274754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-89ea6bc0-badb-49c1-b597-e19ab30abbb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:19.348825 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea6bc0-badb-49c1-b597-e19ab30abbb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:19.350527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea6bc0-badb-49c1-b597-e19ab30abbb2 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:19.361621 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1284/2533] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:42:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:42:20.555363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea11e7-0f6a-478c-bd5c-fbf5a4f9dc78 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:20.565042 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea11e7-0f6a-478c-bd5c-fbf5a4f9dc78 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:20.569354 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb923ab-e448-4854-9a07-2b59d8ecdfab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:20.569936 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb923ab-e448-4854-9a07-2b59d8ecdfab service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:20.570219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb923ab-e448-4854-9a07-2b59d8ecdfab service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:20.570495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb923ab-e448-4854-9a07-2b59d8ecdfab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:20.572274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5fb923ab-e448-4854-9a07-2b59d8ecdfab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:20.586028 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb923ab-e448-4854-9a07-2b59d8ecdfab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:20.587448 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb923ab-e448-4854-9a07-2b59d8ecdfab service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:20.598414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1250/2534] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:42:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:42:29.356664 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=34 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:42:30.593534 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=31 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:42:37.936921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea6bc0-badb-49c1-b597-e19ab30abbb2 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:37.938321 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-89ea6bc0-badb-49c1-b597-e19ab30abbb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:38.004222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea6bc0-badb-49c1-b597-e19ab30abbb2 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:38.008149 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec972a-9578-420a-8a79-f87154809978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:38.008760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec972a-9578-420a-8a79-f87154809978 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:38.009040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec972a-9578-420a-8a79-f87154809978 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:38.009335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec972a-9578-420a-8a79-f87154809978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:38.011637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3cec972a-9578-420a-8a79-f87154809978 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:38.023165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec972a-9578-420a-8a79-f87154809978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:38.024601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec972a-9578-420a-8a79-f87154809978 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:38.033829 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1285/2535] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:42:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:42:43.716386 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb923ab-e448-4854-9a07-2b59d8ecdfab service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:43.717992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5fb923ab-e448-4854-9a07-2b59d8ecdfab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:43.788734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb923ab-e448-4854-9a07-2b59d8ecdfab service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:43.793508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb1f0e-179e-45d0-ba87-0bcdcd1815e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:43.794200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb1f0e-179e-45d0-ba87-0bcdcd1815e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:43.794548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb1f0e-179e-45d0-ba87-0bcdcd1815e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:43.794872 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb1f0e-179e-45d0-ba87-0bcdcd1815e0 admin admin] PATH_INFO: `/v3/users/def1da13ff2a47ada8e5f453061e3f7c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:43.795995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb1f0e-179e-45d0-ba87-0bcdcd1815e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=def1da13ff2a47ada8e5f453061e3f7c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:43.797953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb1f0e-179e-45d0-ba87-0bcdcd1815e0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:43.989976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-29fb1f0e-179e-45d0-ba87-0bcdcd1815e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def1da13ff2a47ada8e5f453061e3f7c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:44.079454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-29fb1f0e-179e-45d0-ba87-0bcdcd1815e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def1da13ff2a47ada8e5f453061e3f7c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:44.092320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-29fb1f0e-179e-45d0-ba87-0bcdcd1815e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def1da13ff2a47ada8e5f453061e3f7c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:44.092985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-29fb1f0e-179e-45d0-ba87-0bcdcd1815e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def1da13ff2a47ada8e5f453061e3f7c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:44.093605 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-29fb1f0e-179e-45d0-ba87-0bcdcd1815e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def1da13ff2a47ada8e5f453061e3f7c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:44.111404 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1251/2536] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:42:43 2022] DELETE /identity/v3/users/def1da13ff2a47ada8e5f453061e3f7c => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:42:44.367033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec972a-9578-420a-8a79-f87154809978 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:44.380621 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec972a-9578-420a-8a79-f87154809978 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:44.450075 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:44.450783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:44.451064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:44.451336 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] PATH_INFO: `/v3/projects/5eea609cee794549a2c23cdb24fbb572` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:44.452499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eea609cee794549a2c23cdb24fbb572)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:44.454420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:44.572343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eea609cee794549a2c23cdb24fbb572', 'request_id': 'req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:44.579375 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eea609cee794549a2c23cdb24fbb572', 'request_id': 'req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:44.683109 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eea609cee794549a2c23cdb24fbb572', 'request_id': 'req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:44.698358 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] The token cache is being invalidate because project 5eea609cee794549a2c23cdb24fbb572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:42:44.698749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:44.699482 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1a06cf36-77ff-4e43-a5b5-c88fccd21a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:44.701439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1286/2537] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:42:44 2022] DELETE /identity/v3/projects/5eea609cee794549a2c23cdb24fbb572 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:42:44.706310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683983d-00fb-410d-bb47-c59a5d6e16e7 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:44.706523 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683983d-00fb-410d-bb47-c59a5d6e16e7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:44.706684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683983d-00fb-410d-bb47-c59a5d6e16e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:44.731954 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-5683983d-00fb-410d-bb47-c59a5d6e16e7 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:42:44.736474 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5683983d-00fb-410d-bb47-c59a5d6e16e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:44.817494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1252/2538] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:42:44.825870 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e14a3-de14-4d8c-a2bc-1cdc9e85756d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:44.826186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e14a3-de14-4d8c-a2bc-1cdc9e85756d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:44.826467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e14a3-de14-4d8c-a2bc-1cdc9e85756d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:44.854444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-666e14a3-de14-4d8c-a2bc-1cdc9e85756d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:42:44.856062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-666e14a3-de14-4d8c-a2bc-1cdc9e85756d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:44.865838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1287/2539] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:42:44.870275 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683983d-00fb-410d-bb47-c59a5d6e16e7 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:44.880711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683983d-00fb-410d-bb47-c59a5d6e16e7 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:44.886742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fc395-ad20-419a-af77-e7745a0d90f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:44.887357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fc395-ad20-419a-af77-e7745a0d90f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:44.887711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fc395-ad20-419a-af77-e7745a0d90f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:44.887984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fc395-ad20-419a-af77-e7745a0d90f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:44.888691 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fc395-ad20-419a-af77-e7745a0d90f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:42:44.888987 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fc395-ad20-419a-af77-e7745a0d90f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:44.890625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fc395-ad20-419a-af77-e7745a0d90f4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:44.903246 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1253/2540] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:42:44.907685 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e14a3-de14-4d8c-a2bc-1cdc9e85756d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:44.917794 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e14a3-de14-4d8c-a2bc-1cdc9e85756d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:44.922603 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849c79e-f1f0-49b0-88d3-c49e7a7cec6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:44.923302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849c79e-f1f0-49b0-88d3-c49e7a7cec6a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:44.928601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849c79e-f1f0-49b0-88d3-c49e7a7cec6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:44.928601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849c79e-f1f0-49b0-88d3-c49e7a7cec6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:44.928601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849c79e-f1f0-49b0-88d3-c49e7a7cec6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:44.928601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849c79e-f1f0-49b0-88d3-c49e7a7cec6a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:45.013430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1288/2541] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:42:44 2022] POST /identity/v3/projects => generated 397 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:42:45.017669 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fc395-ad20-419a-af77-e7745a0d90f4 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:45.027862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fc395-ad20-419a-af77-e7745a0d90f4 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:45.031857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:42:45.090337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e68f51-d809-4ef3-a814-0bbbb632e514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:45.090994 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e68f51-d809-4ef3-a814-0bbbb632e514 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:45.091274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e68f51-d809-4ef3-a814-0bbbb632e514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:45.091548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e68f51-d809-4ef3-a814-0bbbb632e514 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:45.092392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e68f51-d809-4ef3-a814-0bbbb632e514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:45.094328 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e68f51-d809-4ef3-a814-0bbbb632e514 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:45.166337 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1254/2542] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:42:45 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:42:45.170704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849c79e-f1f0-49b0-88d3-c49e7a7cec6a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:45.177919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:42:45.182177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849c79e-f1f0-49b0-88d3-c49e7a7cec6a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:45.187464 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ffdcd-3e5a-415f-b615-1f2509578a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:45.188153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ffdcd-3e5a-415f-b615-1f2509578a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:45.188436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ffdcd-3e5a-415f-b615-1f2509578a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:45.188714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ffdcd-3e5a-415f-b615-1f2509578a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:45.189366 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ffdcd-3e5a-415f-b615-1f2509578a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:45.190973 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ffdcd-3e5a-415f-b615-1f2509578a4c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:45.203076 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1289/2543] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:42:45.207302 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e68f51-d809-4ef3-a814-0bbbb632e514 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:45.218561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e68f51-d809-4ef3-a814-0bbbb632e514 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:45.225310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260962e9-b262-4ef9-945b-58b424de8ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:45.226241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260962e9-b262-4ef9-945b-58b424de8ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:45.226580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260962e9-b262-4ef9-945b-58b424de8ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:45.226861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260962e9-b262-4ef9-945b-58b424de8ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:45.230327 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260962e9-b262-4ef9-945b-58b424de8ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:45.230327 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260962e9-b262-4ef9-945b-58b424de8ba4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:45.241122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1255/2544] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:42:45.245869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ffdcd-3e5a-415f-b615-1f2509578a4c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:45.256231 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ffdcd-3e5a-415f-b615-1f2509578a4c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:45.262166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d7c08-2297-4e2c-b628-eea586121d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:45.262842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d7c08-2297-4e2c-b628-eea586121d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:45.263123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d7c08-2297-4e2c-b628-eea586121d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:45.263400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d7c08-2297-4e2c-b628-eea586121d75 admin admin] PATH_INFO: `/v3/projects/62b86cee10c74b098416f94cb6e0b946/users/d6552e069eea4198b8bd762e9acfff9c/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:45.279413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d7c08-2297-4e2c-b628-eea586121d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b86cee10c74b098416f94cb6e0b946, user_id=d6552e069eea4198b8bd762e9acfff9c, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:45.282129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d7c08-2297-4e2c-b628-eea586121d75 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:45.360150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1290/2545] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:42:45 2022] PUT /identity/v3/projects/62b86cee10c74b098416f94cb6e0b946/users/d6552e069eea4198b8bd762e9acfff9c/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:42:45.371767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c78463-74d0-4800-b802-5639d711f207 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:45.372085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c78463-74d0-4800-b802-5639d711f207 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:45.372374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c78463-74d0-4800-b802-5639d711f207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:45.401273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a2c78463-74d0-4800-b802-5639d711f207 None None] MFA Rules not processed for user `d6552e069eea4198b8bd762e9acfff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:42:45.402538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a2c78463-74d0-4800-b802-5639d711f207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:45.480053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1256/2546] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:42:45 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:42:45.491064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d7c08-2297-4e2c-b628-eea586121d75 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:45.493725 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-105d7c08-2297-4e2c-b628-eea586121d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:45.559773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d7c08-2297-4e2c-b628-eea586121d75 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:45.563492 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751e599-daef-4e1d-a14b-04ab40b7e87f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:45.564130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751e599-daef-4e1d-a14b-04ab40b7e87f service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:45.564413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751e599-daef-4e1d-a14b-04ab40b7e87f service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:45.564718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751e599-daef-4e1d-a14b-04ab40b7e87f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:45.574020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751e599-daef-4e1d-a14b-04ab40b7e87f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:45.575609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751e599-daef-4e1d-a14b-04ab40b7e87f service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:45.586865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1291/2547] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:42:45 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:42:54.598031 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c78463-74d0-4800-b802-5639d711f207 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:54.661760 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c78463-74d0-4800-b802-5639d711f207 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:54.667283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:54.667928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:54.668235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:54.668509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] PATH_INFO: `/v3/users/d6552e069eea4198b8bd762e9acfff9c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:54.669631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6552e069eea4198b8bd762e9acfff9c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:54.671546 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:54.687151 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:42:54.899787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6552e069eea4198b8bd762e9acfff9c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:54.978440 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6552e069eea4198b8bd762e9acfff9c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:54.992199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6552e069eea4198b8bd762e9acfff9c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:54.993012 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6552e069eea4198b8bd762e9acfff9c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:54.993627 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6552e069eea4198b8bd762e9acfff9c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:55.006396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1257/2548] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:42:54 2022] DELETE /identity/v3/users/d6552e069eea4198b8bd762e9acfff9c => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:42:55.011352 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5271a-7335-4385-b305-63ac849d4922 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:55.015051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5271a-7335-4385-b305-63ac849d4922 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:55.015051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5271a-7335-4385-b305-63ac849d4922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:55.031997 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-13d5271a-7335-4385-b305-63ac849d4922 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:42:55.033328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-13d5271a-7335-4385-b305-63ac849d4922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:55.097385 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1292/2549] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:42:55.107759 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:55.109250 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:55.180500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ffdb8-4dd8-4d4a-a642-0d87f09eb43a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:55.184132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c0397-537f-47de-89f9-ee24bcb9a330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:55.184739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c0397-537f-47de-89f9-ee24bcb9a330 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:55.185020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c0397-537f-47de-89f9-ee24bcb9a330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:55.185311 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c0397-537f-47de-89f9-ee24bcb9a330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:55.195525 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c0397-537f-47de-89f9-ee24bcb9a330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:55.197178 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c0397-537f-47de-89f9-ee24bcb9a330 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:55.206966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1258/2550] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:42:55.667967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5271a-7335-4385-b305-63ac849d4922 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:55.669532 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-13d5271a-7335-4385-b305-63ac849d4922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:55.683512 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5271a-7335-4385-b305-63ac849d4922 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:55.688637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:55.689347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:55.689696 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:55.690039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] PATH_INFO: `/v3/projects/62b86cee10c74b098416f94cb6e0b946` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:55.691539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62b86cee10c74b098416f94cb6e0b946)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:55.693617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:55.711824 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c0397-537f-47de-89f9-ee24bcb9a330 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:55.714104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-208c0397-537f-47de-89f9-ee24bcb9a330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:55.798401 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c0397-537f-47de-89f9-ee24bcb9a330 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:55.803890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e90d1-1d66-4bcb-ad30-d5c870523ec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:55.809702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e90d1-1d66-4bcb-ad30-d5c870523ec2 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:55.809702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e90d1-1d66-4bcb-ad30-d5c870523ec2 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:55.809702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e90d1-1d66-4bcb-ad30-d5c870523ec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:55.809702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7d1e90d1-1d66-4bcb-ad30-d5c870523ec2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:55.821960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e90d1-1d66-4bcb-ad30-d5c870523ec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:55.823567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e90d1-1d66-4bcb-ad30-d5c870523ec2 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:55.835757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1259/2551] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:42:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:42:55.848628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62b86cee10c74b098416f94cb6e0b946', 'request_id': 'req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:55.855889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62b86cee10c74b098416f94cb6e0b946', 'request_id': 'req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:55.914788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62b86cee10c74b098416f94cb6e0b946', 'request_id': 'req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:55.920202 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:42:55.927507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] The token cache is being invalidate because project 62b86cee10c74b098416f94cb6e0b946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:42:55.927795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:55.928455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f7ed6ef5-1c61-4706-a83e-e47e9acf4a25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:42:55.930676 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1293/2552] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:42:55 2022] DELETE /identity/v3/projects/62b86cee10c74b098416f94cb6e0b946 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:42:55.935572 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b932e-5d4b-44f3-b448-90ee9046322a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:55.935860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b932e-5d4b-44f3-b448-90ee9046322a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:55.936171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b932e-5d4b-44f3-b448-90ee9046322a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:55.956329 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-076b932e-5d4b-44f3-b448-90ee9046322a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:42:55.957532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-076b932e-5d4b-44f3-b448-90ee9046322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:55.977225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:42:56.024050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1260/2553] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:42:56.032610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262944b7-5406-494e-89cf-f177f2335bba None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:56.032610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262944b7-5406-494e-89cf-f177f2335bba None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:56.032610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262944b7-5406-494e-89cf-f177f2335bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:56.056876 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-262944b7-5406-494e-89cf-f177f2335bba None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:42:56.058129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-262944b7-5406-494e-89cf-f177f2335bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:56.066878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1294/2554] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:42:56.071146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b932e-5d4b-44f3-b448-90ee9046322a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:56.081281 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b932e-5d4b-44f3-b448-90ee9046322a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:56.086793 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e631e-0e1e-4ad2-b765-f4bdfabe4231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:56.087447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e631e-0e1e-4ad2-b765-f4bdfabe4231 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:56.087765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e631e-0e1e-4ad2-b765-f4bdfabe4231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:56.088033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e631e-0e1e-4ad2-b765-f4bdfabe4231 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:56.088820 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e631e-0e1e-4ad2-b765-f4bdfabe4231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:42:56.089132 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e631e-0e1e-4ad2-b765-f4bdfabe4231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:56.090801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e631e-0e1e-4ad2-b765-f4bdfabe4231 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:56.104096 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1261/2555] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:42:56.108733 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262944b7-5406-494e-89cf-f177f2335bba None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:56.119543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262944b7-5406-494e-89cf-f177f2335bba None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:56.124854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cf4d1-09f3-45da-b6f7-f84f4d5618e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:56.125649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cf4d1-09f3-45da-b6f7-f84f4d5618e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:56.126008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cf4d1-09f3-45da-b6f7-f84f4d5618e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:56.126403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cf4d1-09f3-45da-b6f7-f84f4d5618e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:56.127165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cf4d1-09f3-45da-b6f7-f84f4d5618e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:56.129232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cf4d1-09f3-45da-b6f7-f84f4d5618e7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:56.213681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1295/2556] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:42:56 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:42:56.217930 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e631e-0e1e-4ad2-b765-f4bdfabe4231 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:56.229129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e631e-0e1e-4ad2-b765-f4bdfabe4231 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:56.288262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e54496-f912-4d7d-b9c9-6c79bb174d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:56.289641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e54496-f912-4d7d-b9c9-6c79bb174d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:56.290034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e54496-f912-4d7d-b9c9-6c79bb174d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:56.290423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e54496-f912-4d7d-b9c9-6c79bb174d72 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:56.291091 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e54496-f912-4d7d-b9c9-6c79bb174d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:56.293066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e54496-f912-4d7d-b9c9-6c79bb174d72 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:56.386120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1262/2557] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:42:56 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:42:56.387444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cf4d1-09f3-45da-b6f7-f84f4d5618e7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:56.397755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cf4d1-09f3-45da-b6f7-f84f4d5618e7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:56.404626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6826fc-fcc3-45e1-8b02-200d2f19a9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:56.405250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6826fc-fcc3-45e1-8b02-200d2f19a9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:56.405529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6826fc-fcc3-45e1-8b02-200d2f19a9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:56.405798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6826fc-fcc3-45e1-8b02-200d2f19a9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:56.406449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6826fc-fcc3-45e1-8b02-200d2f19a9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:56.408092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6826fc-fcc3-45e1-8b02-200d2f19a9b8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:56.420541 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1296/2558] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:42:56.424765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e54496-f912-4d7d-b9c9-6c79bb174d72 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:56.435160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e54496-f912-4d7d-b9c9-6c79bb174d72 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:56.440001 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b2006-d237-49e1-9f16-00a5db47ab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:56.440573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b2006-d237-49e1-9f16-00a5db47ab3e admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:56.440854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b2006-d237-49e1-9f16-00a5db47ab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:56.441192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b2006-d237-49e1-9f16-00a5db47ab3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:56.441835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b2006-d237-49e1-9f16-00a5db47ab3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:56.443489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b2006-d237-49e1-9f16-00a5db47ab3e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:56.455156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1263/2559] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:42:56.459367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6826fc-fcc3-45e1-8b02-200d2f19a9b8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:56.470704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6826fc-fcc3-45e1-8b02-200d2f19a9b8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:56.475462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c68f09-657f-480f-a75d-43822be58399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:56.477970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c68f09-657f-480f-a75d-43822be58399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:56.477970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c68f09-657f-480f-a75d-43822be58399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:56.477970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c68f09-657f-480f-a75d-43822be58399 admin admin] PATH_INFO: `/v3/projects/bcc9a378e7ba4206baaa7bcc683606ca/users/acf297d1f1f244dd997992583189c127/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:56.494360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c68f09-657f-480f-a75d-43822be58399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc9a378e7ba4206baaa7bcc683606ca, user_id=acf297d1f1f244dd997992583189c127, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:56.497281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c68f09-657f-480f-a75d-43822be58399 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:56.572755 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1297/2560] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:42:56 2022] PUT /identity/v3/projects/bcc9a378e7ba4206baaa7bcc683606ca/users/acf297d1f1f244dd997992583189c127/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:42:56.577682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e4ca2-3ab4-4be9-88b1-35687e60cdeb None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:56.577973 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e4ca2-3ab4-4be9-88b1-35687e60cdeb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:56.578252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e4ca2-3ab4-4be9-88b1-35687e60cdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:56.598041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-341e4ca2-3ab4-4be9-88b1-35687e60cdeb None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:42:56.599336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-341e4ca2-3ab4-4be9-88b1-35687e60cdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:56.659573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1264/2561] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:42:56.669450 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c68f09-657f-480f-a75d-43822be58399 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:42:56.671186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-52c68f09-657f-480f-a75d-43822be58399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:42:56.731499 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c68f09-657f-480f-a75d-43822be58399 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:42:56.735118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d203cbe-2334-429e-aa87-c5d7d91a10e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:42:56.735717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d203cbe-2334-429e-aa87-c5d7d91a10e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:42:56.735998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d203cbe-2334-429e-aa87-c5d7d91a10e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:42:56.736300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d203cbe-2334-429e-aa87-c5d7d91a10e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:42:56.745539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d203cbe-2334-429e-aa87-c5d7d91a10e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:42:56.746976 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d203cbe-2334-429e-aa87-c5d7d91a10e6 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:42:56.756817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1298/2562] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:01.998773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8abe2-4a6c-4659-a43e-9d192dd96f8d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:01.998773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8abe2-4a6c-4659-a43e-9d192dd96f8d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:01.998773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8abe2-4a6c-4659-a43e-9d192dd96f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:02.030507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-cbf8abe2-4a6c-4659-a43e-9d192dd96f8d None None] MFA Rules not processed for user `acf297d1f1f244dd997992583189c127`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:02.031896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cbf8abe2-4a6c-4659-a43e-9d192dd96f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:02.052660 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:02.116542 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1265/2563] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:02.118124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d203cbe-2334-429e-aa87-c5d7d91a10e6 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:02.129131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d203cbe-2334-429e-aa87-c5d7d91a10e6 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:02.140879 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e0938-3343-4c06-906e-e5b5726a3594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:02.141726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e0938-3343-4c06-906e-e5b5726a3594 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:02.142085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e0938-3343-4c06-906e-e5b5726a3594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:02.142432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e0938-3343-4c06-906e-e5b5726a3594 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:02.143163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e0938-3343-4c06-906e-e5b5726a3594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:02.145376 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e0938-3343-4c06-906e-e5b5726a3594 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:02.259588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1299/2564] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:43:02 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:43:02.263203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8abe2-4a6c-4659-a43e-9d192dd96f8d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:02.274653 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8abe2-4a6c-4659-a43e-9d192dd96f8d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:02.346090 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6432760-b66b-43ee-b631-07ce9ffd8c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:02.346739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6432760-b66b-43ee-b631-07ce9ffd8c5c admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:02.347020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6432760-b66b-43ee-b631-07ce9ffd8c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:02.347296 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6432760-b66b-43ee-b631-07ce9ffd8c5c admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:02.347942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6432760-b66b-43ee-b631-07ce9ffd8c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:02.349895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6432760-b66b-43ee-b631-07ce9ffd8c5c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:02.457444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1266/2565] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:43:02 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:43:02.461478 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e0938-3343-4c06-906e-e5b5726a3594 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:02.471643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e0938-3343-4c06-906e-e5b5726a3594 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:02.476247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4ab2b-425d-41a5-aceb-659ad466d845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:02.476800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4ab2b-425d-41a5-aceb-659ad466d845 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:02.477075 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4ab2b-425d-41a5-aceb-659ad466d845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:02.477347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4ab2b-425d-41a5-aceb-659ad466d845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:02.477990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4ab2b-425d-41a5-aceb-659ad466d845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:02.479658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4ab2b-425d-41a5-aceb-659ad466d845 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:02.489969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1300/2566] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:02.494745 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6432760-b66b-43ee-b631-07ce9ffd8c5c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:02.504781 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6432760-b66b-43ee-b631-07ce9ffd8c5c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:02.510177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215d5ee-36f9-4655-a0d5-3f627f5891cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:02.511082 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d5ee-36f9-4655-a0d5-3f627f5891cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:02.511364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d5ee-36f9-4655-a0d5-3f627f5891cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:02.511640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d5ee-36f9-4655-a0d5-3f627f5891cc admin admin] PATH_INFO: `/v3/projects/6aaf55537b3846c0a2cf60a710c38003/users/5eb7cd1bc8de49a8a7fc0db68e3eab58/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:02.526543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215d5ee-36f9-4655-a0d5-3f627f5891cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aaf55537b3846c0a2cf60a710c38003, user_id=5eb7cd1bc8de49a8a7fc0db68e3eab58, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:02.529261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215d5ee-36f9-4655-a0d5-3f627f5891cc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:02.635702 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1267/2567] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:43:02 2022] PUT /identity/v3/projects/6aaf55537b3846c0a2cf60a710c38003/users/5eb7cd1bc8de49a8a7fc0db68e3eab58/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:43:02.639717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4ab2b-425d-41a5-aceb-659ad466d845 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:02.651040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4ab2b-425d-41a5-aceb-659ad466d845 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:02.715570 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c6346-424c-466b-b459-3920463b03c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:02.716208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c6346-424c-466b-b459-3920463b03c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:02.716489 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c6346-424c-466b-b459-3920463b03c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:02.716762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c6346-424c-466b-b459-3920463b03c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:02.717398 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c6346-424c-466b-b459-3920463b03c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:02.718983 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c6346-424c-466b-b459-3920463b03c5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:02.720979 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:02.729109 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1301/2568] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:02.733345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215d5ee-36f9-4655-a0d5-3f627f5891cc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:02.743904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215d5ee-36f9-4655-a0d5-3f627f5891cc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:02.748705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea55cb-2bd9-49aa-8df3-c24afb5e9cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:02.749294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea55cb-2bd9-49aa-8df3-c24afb5e9cca admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:02.749571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea55cb-2bd9-49aa-8df3-c24afb5e9cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:02.749847 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea55cb-2bd9-49aa-8df3-c24afb5e9cca admin admin] PATH_INFO: `/v3/projects/6aaf55537b3846c0a2cf60a710c38003/users/5eb7cd1bc8de49a8a7fc0db68e3eab58/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:02.751854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea55cb-2bd9-49aa-8df3-c24afb5e9cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aaf55537b3846c0a2cf60a710c38003, user_id=5eb7cd1bc8de49a8a7fc0db68e3eab58, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:02.754503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea55cb-2bd9-49aa-8df3-c24afb5e9cca admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:02.767295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1268/2569] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:43:02 2022] PUT /identity/v3/projects/6aaf55537b3846c0a2cf60a710c38003/users/5eb7cd1bc8de49a8a7fc0db68e3eab58/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:03.736261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c6346-424c-466b-b459-3920463b03c5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:03.807901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c6346-424c-466b-b459-3920463b03c5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:03.860490 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2555e7-ee3b-4fd4-a909-44d03016141a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:03.861096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2555e7-ee3b-4fd4-a909-44d03016141a service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:03.861396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2555e7-ee3b-4fd4-a909-44d03016141a service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:03.861662 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2555e7-ee3b-4fd4-a909-44d03016141a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:03.863437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-af2555e7-ee3b-4fd4-a909-44d03016141a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:03.929707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2555e7-ee3b-4fd4-a909-44d03016141a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:03.931380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2555e7-ee3b-4fd4-a909-44d03016141a service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:03.947187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1302/2570] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:04.117953 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea55cb-2bd9-49aa-8df3-c24afb5e9cca admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:04.119927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4aea55cb-2bd9-49aa-8df3-c24afb5e9cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:04.229016 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea55cb-2bd9-49aa-8df3-c24afb5e9cca admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:04.244612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af21652-c1a9-4d94-820f-ed57110e44e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:04.245814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af21652-c1a9-4d94-820f-ed57110e44e6 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:04.246289 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af21652-c1a9-4d94-820f-ed57110e44e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:04.246754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af21652-c1a9-4d94-820f-ed57110e44e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:04.249387 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7af21652-c1a9-4d94-820f-ed57110e44e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:04.261369 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:04.342555 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af21652-c1a9-4d94-820f-ed57110e44e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:04.344343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af21652-c1a9-4d94-820f-ed57110e44e6 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:04.365957 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1269/2571] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:09.614310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2555e7-ee3b-4fd4-a909-44d03016141a service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:09.617511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-af2555e7-ee3b-4fd4-a909-44d03016141a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:09.638757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2555e7-ee3b-4fd4-a909-44d03016141a service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:09.647275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:09.650872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:09.651347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:09.651777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] PATH_INFO: `/v3/users/966045fbe69f43b4b1a825da3ad571b9` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:09.662847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=966045fbe69f43b4b1a825da3ad571b9)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:09.665551 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:09.706783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065cdb0-cab8-4dac-bf17-3191cfd3b7f1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:09.707269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065cdb0-cab8-4dac-bf17-3191cfd3b7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:09.707453 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065cdb0-cab8-4dac-bf17-3191cfd3b7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:09.745403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7065cdb0-cab8-4dac-bf17-3191cfd3b7f1 None None] MFA Rules not processed for user `5eb7cd1bc8de49a8a7fc0db68e3eab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:09.747368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7065cdb0-cab8-4dac-bf17-3191cfd3b7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:09.839496 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1270/2572] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:43:09.852384 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065cdb0-cab8-4dac-bf17-3191cfd3b7f1 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:09.854081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7065cdb0-cab8-4dac-bf17-3191cfd3b7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:09.888198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '966045fbe69f43b4b1a825da3ad571b9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:09.924690 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065cdb0-cab8-4dac-bf17-3191cfd3b7f1 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:09.928698 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff31455-557d-4ec9-bb32-93bf04a908ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb2129c006f4ddeba7d128a8d43d5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:09.929875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff31455-557d-4ec9-bb32-93bf04a908ee service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:09.930367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff31455-557d-4ec9-bb32-93bf04a908ee service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:09.930655 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff31455-557d-4ec9-bb32-93bf04a908ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:09.965535 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '966045fbe69f43b4b1a825da3ad571b9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:09.977227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:09.997379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:10.023420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '966045fbe69f43b4b1a825da3ad571b9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:10.039471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff31455-557d-4ec9-bb32-93bf04a908ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:10.041155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff31455-557d-4ec9-bb32-93bf04a908ee service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:10.041763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '966045fbe69f43b4b1a825da3ad571b9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:10.042672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '966045fbe69f43b4b1a825da3ad571b9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:10.055410 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1303/2573] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:43:09 2022] DELETE /identity/v3/users/966045fbe69f43b4b1a825da3ad571b9 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:10.067283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1271/2574] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:10.317216 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:10.320967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:10.338700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b96d39-06b3-467c-ac1b-c123d6aaf758 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:10.338969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b96d39-06b3-467c-ac1b-c123d6aaf758 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:10.339244 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b96d39-06b3-467c-ac1b-c123d6aaf758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:10.368826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-48b96d39-06b3-467c-ac1b-c123d6aaf758 None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:10.370574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-48b96d39-06b3-467c-ac1b-c123d6aaf758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:10.423157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a9192-eb90-44db-a0c3-032906410ac6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:10.433298 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:10.434229 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:10.437168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:10.437168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] PATH_INFO: `/v3/projects/1a78ad9d7151473798fbefa330678250` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:10.437168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a78ad9d7151473798fbefa330678250)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:10.438227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:10.449022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1272/2575] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:43:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:43:10.585070 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a78ad9d7151473798fbefa330678250', 'request_id': 'req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:10.591935 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a78ad9d7151473798fbefa330678250', 'request_id': 'req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:10.606540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b96d39-06b3-467c-ac1b-c123d6aaf758 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:10.607968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-48b96d39-06b3-467c-ac1b-c123d6aaf758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:10.680605 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a78ad9d7151473798fbefa330678250', 'request_id': 'req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:10.686754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b96d39-06b3-467c-ac1b-c123d6aaf758 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:10.693107 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] The token cache is being invalidate because project 1a78ad9d7151473798fbefa330678250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:43:10.693481 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:10.694316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-49cd21a3-1ca4-48d3-bccc-45c1c4238ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:10.696186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1304/2576] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:43:10 2022] DELETE /identity/v3/projects/1a78ad9d7151473798fbefa330678250 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:10.700965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a6d2e-e187-4cd8-ae7e-0f8dd3f7adf8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:10.701247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a6d2e-e187-4cd8-ae7e-0f8dd3f7adf8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:10.701542 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a6d2e-e187-4cd8-ae7e-0f8dd3f7adf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:10.721923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-131a6d2e-e187-4cd8-ae7e-0f8dd3f7adf8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:10.723379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-131a6d2e-e187-4cd8-ae7e-0f8dd3f7adf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:10.749141 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae9c4a-5b9d-42d5-9329-c47c7a15914f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:10.749852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9c4a-5b9d-42d5-9329-c47c7a15914f service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:10.750194 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9c4a-5b9d-42d5-9329-c47c7a15914f service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:10.750537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9c4a-5b9d-42d5-9329-c47c7a15914f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:10.752589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cbae9c4a-5b9d-42d5-9329-c47c7a15914f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:10.791961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1305/2577] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:43:10.794327 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:10.799719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783be57-89ad-4c48-94c3-e8e865d6d0e0 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:10.800031 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783be57-89ad-4c48-94c3-e8e865d6d0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:10.800309 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783be57-89ad-4c48-94c3-e8e865d6d0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:10.815025 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:10.822514 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2783be57-89ad-4c48-94c3-e8e865d6d0e0 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:10.823078 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae9c4a-5b9d-42d5-9329-c47c7a15914f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:10.823792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2783be57-89ad-4c48-94c3-e8e865d6d0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:10.824888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae9c4a-5b9d-42d5-9329-c47c7a15914f service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:10.832808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1306/2578] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:10.833850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1273/2579] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:43:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 07 22:43:10.837194 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783be57-89ad-4c48-94c3-e8e865d6d0e0 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:10.849000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783be57-89ad-4c48-94c3-e8e865d6d0e0 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:10.857264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93514203-8b28-44b5-b976-24e03d557e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:10.858440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93514203-8b28-44b5-b976-24e03d557e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:10.858440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93514203-8b28-44b5-b976-24e03d557e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:10.858963 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93514203-8b28-44b5-b976-24e03d557e60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:10.859826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93514203-8b28-44b5-b976-24e03d557e60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:43:10.860400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93514203-8b28-44b5-b976-24e03d557e60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:10.862365 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93514203-8b28-44b5-b976-24e03d557e60 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:10.877046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1307/2580] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:43:10.880924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae9c4a-5b9d-42d5-9329-c47c7a15914f service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:10.893543 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae9c4a-5b9d-42d5-9329-c47c7a15914f service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:10.898832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a302ad9-81fb-4d69-b735-207f1966a372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:10.899667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a302ad9-81fb-4d69-b735-207f1966a372 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:10.900294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a302ad9-81fb-4d69-b735-207f1966a372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:10.900730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a302ad9-81fb-4d69-b735-207f1966a372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:10.901540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a302ad9-81fb-4d69-b735-207f1966a372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:10.903600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a302ad9-81fb-4d69-b735-207f1966a372 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:10.996660 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1274/2581] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:43:10 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:43:11.000881 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93514203-8b28-44b5-b976-24e03d557e60 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:11.013694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93514203-8b28-44b5-b976-24e03d557e60 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:11.089813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b5b49-0478-412e-9fc0-fa696e68eef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:11.090427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b5b49-0478-412e-9fc0-fa696e68eef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:11.090706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b5b49-0478-412e-9fc0-fa696e68eef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:11.090979 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b5b49-0478-412e-9fc0-fa696e68eef5 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:11.091609 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b5b49-0478-412e-9fc0-fa696e68eef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:11.095335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b5b49-0478-412e-9fc0-fa696e68eef5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:11.187507 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1308/2582] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:43:10 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:43:11.188819 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a302ad9-81fb-4d69-b735-207f1966a372 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:11.204112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a302ad9-81fb-4d69-b735-207f1966a372 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:11.219902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40052d63-ae7b-4da2-a47b-89b511a1c785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:11.220592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052d63-ae7b-4da2-a47b-89b511a1c785 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:11.220932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052d63-ae7b-4da2-a47b-89b511a1c785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:11.221228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052d63-ae7b-4da2-a47b-89b511a1c785 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:11.221884 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40052d63-ae7b-4da2-a47b-89b511a1c785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:11.223489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40052d63-ae7b-4da2-a47b-89b511a1c785 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:11.244575 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1275/2583] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:11.248468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b5b49-0478-412e-9fc0-fa696e68eef5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:11.259270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b5b49-0478-412e-9fc0-fa696e68eef5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:11.270412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347339c5-7d87-4ceb-8345-e894a420f924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:11.271050 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347339c5-7d87-4ceb-8345-e894a420f924 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:11.271331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347339c5-7d87-4ceb-8345-e894a420f924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:11.271703 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347339c5-7d87-4ceb-8345-e894a420f924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:11.272301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347339c5-7d87-4ceb-8345-e894a420f924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:11.274649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347339c5-7d87-4ceb-8345-e894a420f924 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:11.292596 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1309/2584] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:11.296631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40052d63-ae7b-4da2-a47b-89b511a1c785 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:11.309997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40052d63-ae7b-4da2-a47b-89b511a1c785 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:11.315598 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c73dee-fa80-49ee-8008-58d707a8c02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:11.316363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c73dee-fa80-49ee-8008-58d707a8c02d admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:11.316654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c73dee-fa80-49ee-8008-58d707a8c02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:11.316932 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c73dee-fa80-49ee-8008-58d707a8c02d admin admin] PATH_INFO: `/v3/projects/8bec6d4d3a7343219d5cb8708fe41182/users/a9c9746b11594619bb056613ea2091f4/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:11.336867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c73dee-fa80-49ee-8008-58d707a8c02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bec6d4d3a7343219d5cb8708fe41182, user_id=a9c9746b11594619bb056613ea2091f4, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:11.339531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c73dee-fa80-49ee-8008-58d707a8c02d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:11.417514 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1276/2585] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:43:11 2022] PUT /identity/v3/projects/8bec6d4d3a7343219d5cb8708fe41182/users/a9c9746b11594619bb056613ea2091f4/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:43:11.430023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6b2d6-90cc-4b36-9fbd-78f6cc39283b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:11.430660 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6b2d6-90cc-4b36-9fbd-78f6cc39283b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:11.431012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6b2d6-90cc-4b36-9fbd-78f6cc39283b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:11.470703 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c3a6b2d6-90cc-4b36-9fbd-78f6cc39283b None None] MFA Rules not processed for user `a9c9746b11594619bb056613ea2091f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:11.473994 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c3a6b2d6-90cc-4b36-9fbd-78f6cc39283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:11.576563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1310/2586] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:11.589074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c73dee-fa80-49ee-8008-58d707a8c02d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:11.590978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-70c73dee-fa80-49ee-8008-58d707a8c02d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:11.690494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c73dee-fa80-49ee-8008-58d707a8c02d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:11.697532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f8ef4-1037-4599-a554-b4903369fa01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:11.698331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f8ef4-1037-4599-a554-b4903369fa01 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:11.698703 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f8ef4-1037-4599-a554-b4903369fa01 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:11.699069 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f8ef4-1037-4599-a554-b4903369fa01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:11.707950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f8ef4-1037-4599-a554-b4903369fa01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:11.710635 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f8ef4-1037-4599-a554-b4903369fa01 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:11.734008 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1277/2587] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:20.251840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6b2d6-90cc-4b36-9fbd-78f6cc39283b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:20.335438 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:20.341089 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6b2d6-90cc-4b36-9fbd-78f6cc39283b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:20.345580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:20.346195 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:20.346472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:20.346742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] PATH_INFO: `/v3/users/a9c9746b11594619bb056613ea2091f4` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:20.347852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9c9746b11594619bb056613ea2091f4)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:20.349784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:20.717327 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9c9746b11594619bb056613ea2091f4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:20.801342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9c9746b11594619bb056613ea2091f4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:20.823970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9c9746b11594619bb056613ea2091f4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:20.842254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9c9746b11594619bb056613ea2091f4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:20.843066 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9c9746b11594619bb056613ea2091f4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:20.854763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1311/2588] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:43:20 2022] DELETE /identity/v3/users/a9c9746b11594619bb056613ea2091f4 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:20.859014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eee07f-7737-4587-9559-4987b32baebb None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:20.859519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eee07f-7737-4587-9559-4987b32baebb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:20.859519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eee07f-7737-4587-9559-4987b32baebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:20.895676 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f9eee07f-7737-4587-9559-4987b32baebb None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:20.896978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f9eee07f-7737-4587-9559-4987b32baebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:21.010073 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1278/2589] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:21.019020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:21.020724 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:21.090768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295d724-1316-4d01-97c3-01f3d5d3e5ae admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:21.095841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1334b9-6c83-44c3-92fe-24b7fafbdffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:21.096803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1334b9-6c83-44c3-92fe-24b7fafbdffa service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:21.097061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1334b9-6c83-44c3-92fe-24b7fafbdffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:21.097341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1334b9-6c83-44c3-92fe-24b7fafbdffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:21.110271 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1334b9-6c83-44c3-92fe-24b7fafbdffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:21.111811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1334b9-6c83-44c3-92fe-24b7fafbdffa service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:21.123175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1312/2590] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:21.458559 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b525e5-0f3a-4cf9-95ae-ab2ee957ce37 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:21.458843 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b525e5-0f3a-4cf9-95ae-ab2ee957ce37 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:21.459117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b525e5-0f3a-4cf9-95ae-ab2ee957ce37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:21.467524 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:21.478999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a4b525e5-0f3a-4cf9-95ae-ab2ee957ce37 None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:21.480306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a4b525e5-0f3a-4cf9-95ae-ab2ee957ce37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:21.547414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1279/2591] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:43:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:21.565510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1334b9-6c83-44c3-92fe-24b7fafbdffa service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:21.565510 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fd1334b9-6c83-44c3-92fe-24b7fafbdffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:21.639393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1334b9-6c83-44c3-92fe-24b7fafbdffa service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:21.644650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd937e05-d97f-4fcd-b177-22f7c224bc73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:21.645255 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937e05-d97f-4fcd-b177-22f7c224bc73 service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:21.645531 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937e05-d97f-4fcd-b177-22f7c224bc73 service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:21.645883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937e05-d97f-4fcd-b177-22f7c224bc73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:21.655143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd937e05-d97f-4fcd-b177-22f7c224bc73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:21.656781 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd937e05-d97f-4fcd-b177-22f7c224bc73 service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:21.665377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1313/2592] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:43:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:43:21.727005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b525e5-0f3a-4cf9-95ae-ab2ee957ce37 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:21.728479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a4b525e5-0f3a-4cf9-95ae-ab2ee957ce37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:21.743656 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b525e5-0f3a-4cf9-95ae-ab2ee957ce37 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:21.748664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:21.752950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:21.752950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:21.752950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] PATH_INFO: `/v3/projects/8bec6d4d3a7343219d5cb8708fe41182` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:21.752950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bec6d4d3a7343219d5cb8708fe41182)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:21.754036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:21.863443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bec6d4d3a7343219d5cb8708fe41182', 'request_id': 'req-fde70bdb-c68e-4752-a73b-fdee2106e774'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:21.872522 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bec6d4d3a7343219d5cb8708fe41182', 'request_id': 'req-fde70bdb-c68e-4752-a73b-fdee2106e774'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:21.939402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391b51-0572-443e-8d4f-84412cece379 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:21.939402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391b51-0572-443e-8d4f-84412cece379 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:21.939402 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391b51-0572-443e-8d4f-84412cece379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:21.950747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bec6d4d3a7343219d5cb8708fe41182', 'request_id': 'req-fde70bdb-c68e-4752-a73b-fdee2106e774'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:21.955854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:21.968594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] The token cache is being invalidate because project 8bec6d4d3a7343219d5cb8708fe41182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:43:21.969277 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:21.970243 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fde70bdb-c68e-4752-a73b-fdee2106e774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:21.977174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-50391b51-0572-443e-8d4f-84412cece379 None None] MFA Rules not processed for user `acf297d1f1f244dd997992583189c127`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:21.977396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1280/2593] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:43:21 2022] DELETE /identity/v3/projects/8bec6d4d3a7343219d5cb8708fe41182 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:21.978092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-50391b51-0572-443e-8d4f-84412cece379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:21.979588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa9723-7859-406b-a0b8-e24c28eb4ef8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:21.980133 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa9723-7859-406b-a0b8-e24c28eb4ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:21.981268 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa9723-7859-406b-a0b8-e24c28eb4ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:22.022338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-02fa9723-7859-406b-a0b8-e24c28eb4ef8 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:22.024670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-02fa9723-7859-406b-a0b8-e24c28eb4ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:22.110798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1281/2594] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:22.119547 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a305-3bd0-4a54-b987-bf80f1d97945 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:22.120094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a305-3bd0-4a54-b987-bf80f1d97945 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:22.120487 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a305-3bd0-4a54-b987-bf80f1d97945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:22.134206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1314/2595] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:43:22.154014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3f09a305-3bd0-4a54-b987-bf80f1d97945 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:22.161542 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3f09a305-3bd0-4a54-b987-bf80f1d97945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:22.183850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1282/2596] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:22.189933 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50391b51-0572-443e-8d4f-84412cece379 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:22.205288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50391b51-0572-443e-8d4f-84412cece379 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:22.213201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e84fd-def1-401d-b07b-4475cd4f8fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:22.213837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e84fd-def1-401d-b07b-4475cd4f8fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:22.214115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e84fd-def1-401d-b07b-4475cd4f8fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:22.214387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e84fd-def1-401d-b07b-4475cd4f8fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:22.215064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e84fd-def1-401d-b07b-4475cd4f8fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:43:22.215373 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e84fd-def1-401d-b07b-4475cd4f8fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:22.222858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e84fd-def1-401d-b07b-4475cd4f8fa8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:22.252049 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1315/2597] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:43:22.256426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09a305-3bd0-4a54-b987-bf80f1d97945 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:22.269302 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09a305-3bd0-4a54-b987-bf80f1d97945 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:22.275629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c682e-63c3-45d9-94d1-f8eeb162b60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:22.276292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c682e-63c3-45d9-94d1-f8eeb162b60d admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:22.276574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c682e-63c3-45d9-94d1-f8eeb162b60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:22.276848 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c682e-63c3-45d9-94d1-f8eeb162b60d admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:22.277503 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c682e-63c3-45d9-94d1-f8eeb162b60d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:22.280463 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c682e-63c3-45d9-94d1-f8eeb162b60d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:22.386491 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1283/2598] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:43:22 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:43:22.389736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e84fd-def1-401d-b07b-4475cd4f8fa8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:22.419519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e84fd-def1-401d-b07b-4475cd4f8fa8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:22.509432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefc392-6cb2-4597-85fc-1cfd5cbbeb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:22.511287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefc392-6cb2-4597-85fc-1cfd5cbbeb34 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:22.511580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefc392-6cb2-4597-85fc-1cfd5cbbeb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:22.511858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefc392-6cb2-4597-85fc-1cfd5cbbeb34 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:22.513036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefc392-6cb2-4597-85fc-1cfd5cbbeb34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:22.515561 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefc392-6cb2-4597-85fc-1cfd5cbbeb34 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:22.614771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c682e-63c3-45d9-94d1-f8eeb162b60d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:22.616997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-466c682e-63c3-45d9-94d1-f8eeb162b60d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:22.634224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:22.650093 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1316/2599] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:43:22 2022] POST /identity/v3/users => generated 284 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:43:22.651535 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefc392-6cb2-4597-85fc-1cfd5cbbeb34 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:22.666580 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefc392-6cb2-4597-85fc-1cfd5cbbeb34 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:22.672286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7256d-73fb-49f3-9ccf-32e6b22d74e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:22.673516 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7256d-73fb-49f3-9ccf-32e6b22d74e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:22.673850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7256d-73fb-49f3-9ccf-32e6b22d74e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:22.674222 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7256d-73fb-49f3-9ccf-32e6b22d74e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:22.675254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7256d-73fb-49f3-9ccf-32e6b22d74e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:22.677026 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7256d-73fb-49f3-9ccf-32e6b22d74e0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:22.692598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1317/2600] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:22.698317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7256d-73fb-49f3-9ccf-32e6b22d74e0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:22.710627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7256d-73fb-49f3-9ccf-32e6b22d74e0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:22.716557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cf431-8b6c-42b4-be61-b07634a5dddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:22.717166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cf431-8b6c-42b4-be61-b07634a5dddf admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:22.717765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cf431-8b6c-42b4-be61-b07634a5dddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:22.717765 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cf431-8b6c-42b4-be61-b07634a5dddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:22.718443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cf431-8b6c-42b4-be61-b07634a5dddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:22.720007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cf431-8b6c-42b4-be61-b07634a5dddf admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:22.735495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1318/2601] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:22.735495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cf431-8b6c-42b4-be61-b07634a5dddf admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:22.737539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-149cf431-8b6c-42b4-be61-b07634a5dddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:22.776959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c682e-63c3-45d9-94d1-f8eeb162b60d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:22.782838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb6e6f-2632-4d79-8dca-aa6d9be2bc36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:22.783458 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb6e6f-2632-4d79-8dca-aa6d9be2bc36 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:22.783730 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb6e6f-2632-4d79-8dca-aa6d9be2bc36 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:22.784000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb6e6f-2632-4d79-8dca-aa6d9be2bc36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:22.785828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-abbb6e6f-2632-4d79-8dca-aa6d9be2bc36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:22.806330 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb6e6f-2632-4d79-8dca-aa6d9be2bc36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:22.807444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cf431-8b6c-42b4-be61-b07634a5dddf admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:22.808262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb6e6f-2632-4d79-8dca-aa6d9be2bc36 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:22.811310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739da42-0ca0-48e2-b8ff-0745975c3b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:22.812204 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739da42-0ca0-48e2-b8ff-0745975c3b80 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:22.812563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739da42-0ca0-48e2-b8ff-0745975c3b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:22.813779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739da42-0ca0-48e2-b8ff-0745975c3b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:22.816643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4739da42-0ca0-48e2-b8ff-0745975c3b80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:22.829341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739da42-0ca0-48e2-b8ff-0745975c3b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:22.830860 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739da42-0ca0-48e2-b8ff-0745975c3b80 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:22.833789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1284/2602] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:22.833789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb6e6f-2632-4d79-8dca-aa6d9be2bc36 service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:22.842894 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1319/2603] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:22.854354 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb6e6f-2632-4d79-8dca-aa6d9be2bc36 service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:22.864052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a8c21-475b-405d-8341-3c7e9516aebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:22.865516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a8c21-475b-405d-8341-3c7e9516aebf admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:22.865516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a8c21-475b-405d-8341-3c7e9516aebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:22.865516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a8c21-475b-405d-8341-3c7e9516aebf admin admin] PATH_INFO: `/v3/projects/bbe3f48eee8e4e81870753da5edae61a/users/74dd5e81330740cb9b37c3473b384f06/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:22.887868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a8c21-475b-405d-8341-3c7e9516aebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe3f48eee8e4e81870753da5edae61a, user_id=74dd5e81330740cb9b37c3473b384f06, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:22.891009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a8c21-475b-405d-8341-3c7e9516aebf admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:22.982390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1285/2604] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:43:22 2022] PUT /identity/v3/projects/bbe3f48eee8e4e81870753da5edae61a/users/74dd5e81330740cb9b37c3473b384f06/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:22.994263 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36426455-e061-4318-9e97-61f84f936966 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:22.994640 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36426455-e061-4318-9e97-61f84f936966 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:22.994988 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36426455-e061-4318-9e97-61f84f936966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:23.019265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:23.030474 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-36426455-e061-4318-9e97-61f84f936966 None None] MFA Rules not processed for user `74dd5e81330740cb9b37c3473b384f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:23.031628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-36426455-e061-4318-9e97-61f84f936966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:23.140952 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1320/2605] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:23.145646 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a8c21-475b-405d-8341-3c7e9516aebf admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:23.224137 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a8c21-475b-405d-8341-3c7e9516aebf admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:23.229882 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4190a4-3005-42c0-883c-a5c0fd0691d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:23.230489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4190a4-3005-42c0-883c-a5c0fd0691d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:23.230768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4190a4-3005-42c0-883c-a5c0fd0691d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:23.231117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4190a4-3005-42c0-883c-a5c0fd0691d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:23.231757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4190a4-3005-42c0-883c-a5c0fd0691d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:23.235608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4190a4-3005-42c0-883c-a5c0fd0691d8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:23.300627 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36426455-e061-4318-9e97-61f84f936966 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:23.305409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-36426455-e061-4318-9e97-61f84f936966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:23.334787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1286/2606] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:43:23 2022] POST /identity/v3/projects => generated 367 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:43:23.339342 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4190a4-3005-42c0-883c-a5c0fd0691d8 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:23.417386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36426455-e061-4318-9e97-61f84f936966 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:23.419569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:23.430988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4190a4-3005-42c0-883c-a5c0fd0691d8 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:23.441388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a133-324a-4647-bf69-d870caad1675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:23.442019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a133-324a-4647-bf69-d870caad1675 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:23.442369 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a133-324a-4647-bf69-d870caad1675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:23.442630 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a133-324a-4647-bf69-d870caad1675 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:23.443274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83a133-324a-4647-bf69-d870caad1675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:23.445238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83a133-324a-4647-bf69-d870caad1675 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:23.492395 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2210e-e170-4566-8ca7-b617a130f5a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:23.493016 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2210e-e170-4566-8ca7-b617a130f5a1 service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:23.493296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2210e-e170-4566-8ca7-b617a130f5a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:23.493623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2210e-e170-4566-8ca7-b617a130f5a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:23.495598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9df2210e-e170-4566-8ca7-b617a130f5a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:23.509386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2210e-e170-4566-8ca7-b617a130f5a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:23.511367 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2210e-e170-4566-8ca7-b617a130f5a1 service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:23.522532 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1321/2607] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:23.555259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1287/2608] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:43:23 2022] POST /identity/v3/users => generated 290 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:43:23.556424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2210e-e170-4566-8ca7-b617a130f5a1 service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:23.574520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2210e-e170-4566-8ca7-b617a130f5a1 service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:23.583568 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2212c84-6a99-4451-88e1-ab4e87b914ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:23.584219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2212c84-6a99-4451-88e1-ab4e87b914ef admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:23.584517 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2212c84-6a99-4451-88e1-ab4e87b914ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:23.584768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2212c84-6a99-4451-88e1-ab4e87b914ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:23.586588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2212c84-6a99-4451-88e1-ab4e87b914ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:23.588165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2212c84-6a99-4451-88e1-ab4e87b914ef admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:23.608166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1322/2609] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:23.610022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a133-324a-4647-bf69-d870caad1675 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:23.625284 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a133-324a-4647-bf69-d870caad1675 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:23.635861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7ee07-b7f4-4be3-bc61-da02b5aa849c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:23.636623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7ee07-b7f4-4be3-bc61-da02b5aa849c admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:23.643695 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7ee07-b7f4-4be3-bc61-da02b5aa849c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:23.643695 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7ee07-b7f4-4be3-bc61-da02b5aa849c admin admin] PATH_INFO: `/v3/projects/9ec172047d694c48a54021b00aa153ea/users/6067b880c0874a8294160e802a8552cf/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:23.676268 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7ee07-b7f4-4be3-bc61-da02b5aa849c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec172047d694c48a54021b00aa153ea, user_id=6067b880c0874a8294160e802a8552cf, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:23.681348 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7ee07-b7f4-4be3-bc61-da02b5aa849c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:23.763060 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1288/2610] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:43:23 2022] PUT /identity/v3/projects/9ec172047d694c48a54021b00aa153ea/users/6067b880c0874a8294160e802a8552cf/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:23.767134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2212c84-6a99-4451-88e1-ab4e87b914ef admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:23.779631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2212c84-6a99-4451-88e1-ab4e87b914ef admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:23.810327 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:23.897489 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ece2a8-78f4-4876-b1f2-e5581e3bb5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:23.898097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ece2a8-78f4-4876-b1f2-e5581e3bb5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:23.898925 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ece2a8-78f4-4876-b1f2-e5581e3bb5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:23.899170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ece2a8-78f4-4876-b1f2-e5581e3bb5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:23.899925 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ece2a8-78f4-4876-b1f2-e5581e3bb5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:23.901658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ece2a8-78f4-4876-b1f2-e5581e3bb5d5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:23.916067 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1323/2611] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:23.919800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7ee07-b7f4-4be3-bc61-da02b5aa849c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:23.936250 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7ee07-b7f4-4be3-bc61-da02b5aa849c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:23.943209 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554984-b0ef-4bb5-88a4-43a9c7425b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:23.944466 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554984-b0ef-4bb5-88a4-43a9c7425b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:23.945275 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554984-b0ef-4bb5-88a4-43a9c7425b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:23.946452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554984-b0ef-4bb5-88a4-43a9c7425b95 admin admin] PATH_INFO: `/v3/projects/9ec172047d694c48a54021b00aa153ea/users/6067b880c0874a8294160e802a8552cf/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:23.953439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88554984-b0ef-4bb5-88a4-43a9c7425b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec172047d694c48a54021b00aa153ea, user_id=6067b880c0874a8294160e802a8552cf, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:23.956175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88554984-b0ef-4bb5-88a4-43a9c7425b95 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:23.970216 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1289/2612] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:43:23 2022] PUT /identity/v3/projects/9ec172047d694c48a54021b00aa153ea/users/6067b880c0874a8294160e802a8552cf/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:23.981617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a50d76-2c1e-44f5-b0a4-df55977e8b1c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:23.981925 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a50d76-2c1e-44f5-b0a4-df55977e8b1c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:23.982314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a50d76-2c1e-44f5-b0a4-df55977e8b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:24.018957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-21a50d76-2c1e-44f5-b0a4-df55977e8b1c None None] MFA Rules not processed for user `6067b880c0874a8294160e802a8552cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:24.021011 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-21a50d76-2c1e-44f5-b0a4-df55977e8b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:24.122640 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1324/2613] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:24.137024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554984-b0ef-4bb5-88a4-43a9c7425b95 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:24.140782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-88554984-b0ef-4bb5-88a4-43a9c7425b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:24.219811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554984-b0ef-4bb5-88a4-43a9c7425b95 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:24.223668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cdfd4-e6a9-4c27-a7c1-ef523620f89f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:24.224354 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cdfd4-e6a9-4c27-a7c1-ef523620f89f service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:24.224640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cdfd4-e6a9-4c27-a7c1-ef523620f89f service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:24.224917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cdfd4-e6a9-4c27-a7c1-ef523620f89f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:24.238306 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627cdfd4-e6a9-4c27-a7c1-ef523620f89f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:24.239993 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627cdfd4-e6a9-4c27-a7c1-ef523620f89f service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:24.252058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1290/2614] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:43:24.375371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a50d76-2c1e-44f5-b0a4-df55977e8b1c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:24.377001 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-21a50d76-2c1e-44f5-b0a4-df55977e8b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:24.401628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a50d76-2c1e-44f5-b0a4-df55977e8b1c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:24.426197 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c293604-c8ab-49fd-83b3-08e9f0c868aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:24.426939 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c293604-c8ab-49fd-83b3-08e9f0c868aa service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:24.427289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c293604-c8ab-49fd-83b3-08e9f0c868aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:24.427797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c293604-c8ab-49fd-83b3-08e9f0c868aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:24.541393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c293604-c8ab-49fd-83b3-08e9f0c868aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:24.542962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c293604-c8ab-49fd-83b3-08e9f0c868aa service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:24.554971 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1325/2615] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:27.349997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425822e-f639-44dd-8577-dafcd06404b9 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:27.350582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425822e-f639-44dd-8577-dafcd06404b9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:27.351037 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425822e-f639-44dd-8577-dafcd06404b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:27.413276 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7425822e-f639-44dd-8577-dafcd06404b9 None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:27.415291 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7425822e-f639-44dd-8577-dafcd06404b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:27.496178 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1291/2616] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:43:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:27.512261 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c293604-c8ab-49fd-83b3-08e9f0c868aa service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:27.513722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2c293604-c8ab-49fd-83b3-08e9f0c868aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:27.590917 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:27.632529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c293604-c8ab-49fd-83b3-08e9f0c868aa service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:27.640672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f516b28-9430-4678-b043-36a417ccdea4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:27.641428 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f516b28-9430-4678-b043-36a417ccdea4 service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:27.641704 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f516b28-9430-4678-b043-36a417ccdea4 service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:27.641987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f516b28-9430-4678-b043-36a417ccdea4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:27.655556 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f516b28-9430-4678-b043-36a417ccdea4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:27.657536 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f516b28-9430-4678-b043-36a417ccdea4 service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:27.678553 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1326/2617] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:43:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:43:28.375699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425822e-f639-44dd-8577-dafcd06404b9 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:28.558845 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425822e-f639-44dd-8577-dafcd06404b9 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:28.563777 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503f048-e2be-4552-8a5f-acb9138ea70a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:28.564571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503f048-e2be-4552-8a5f-acb9138ea70a service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:28.564923 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503f048-e2be-4552-8a5f-acb9138ea70a service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:28.565273 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503f048-e2be-4552-8a5f-acb9138ea70a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:28.567268 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7503f048-e2be-4552-8a5f-acb9138ea70a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:28.574535 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:28.578390 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503f048-e2be-4552-8a5f-acb9138ea70a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:28.579826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503f048-e2be-4552-8a5f-acb9138ea70a service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:28.588725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1292/2618] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:30.825498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f516b28-9430-4678-b043-36a417ccdea4 service swift] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:30.959448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f516b28-9430-4678-b043-36a417ccdea4 service swift] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:30.966807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:30.967996 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:30.968634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:30.969279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] PATH_INFO: `/v3/users/74dd5e81330740cb9b37c3473b384f06` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:30.976060 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74dd5e81330740cb9b37c3473b384f06)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:30.979534 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:31.248328 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74dd5e81330740cb9b37c3473b384f06'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:31.845801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74dd5e81330740cb9b37c3473b384f06'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:31.871744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74dd5e81330740cb9b37c3473b384f06'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:31.888281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74dd5e81330740cb9b37c3473b384f06'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:31.894219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74dd5e81330740cb9b37c3473b384f06'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:31.905703 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1327/2619] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:43:30 2022] DELETE /identity/v3/users/74dd5e81330740cb9b37c3473b384f06 => generated 0 bytes in 1078 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:31.906892 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fc4cd-79cc-4bea-9c01-2ba19948442d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:31.907436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fc4cd-79cc-4bea-9c01-2ba19948442d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:31.907561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fc4cd-79cc-4bea-9c01-2ba19948442d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:31.934201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ed8fc4cd-79cc-4bea-9c01-2ba19948442d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:31.939821 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ed8fc4cd-79cc-4bea-9c01-2ba19948442d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:32.080053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1293/2620] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:32.089940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:32.092773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:32.291145 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d3223-2d10-4e71-a3e0-032c72763de5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:32.293477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:32.307957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668ec0a-0ceb-4589-bab3-df3ff9a6071f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:32.307957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668ec0a-0ceb-4589-bab3-df3ff9a6071f service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:32.307957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668ec0a-0ceb-4589-bab3-df3ff9a6071f service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:32.307957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668ec0a-0ceb-4589-bab3-df3ff9a6071f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:32.338938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668ec0a-0ceb-4589-bab3-df3ff9a6071f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:32.340563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668ec0a-0ceb-4589-bab3-df3ff9a6071f service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:32.362939 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1328/2621] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:33.859374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fc4cd-79cc-4bea-9c01-2ba19948442d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:33.861225 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ed8fc4cd-79cc-4bea-9c01-2ba19948442d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:33.873864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fc4cd-79cc-4bea-9c01-2ba19948442d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:33.880539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:33.885564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:33.885564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:33.893395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] PATH_INFO: `/v3/projects/bbe3f48eee8e4e81870753da5edae61a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:33.895146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbe3f48eee8e4e81870753da5edae61a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:33.897392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:34.024004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe3f48eee8e4e81870753da5edae61a', 'request_id': 'req-8544c221-358c-48e6-898e-cbb5bc05f05f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:34.034572 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe3f48eee8e4e81870753da5edae61a', 'request_id': 'req-8544c221-358c-48e6-898e-cbb5bc05f05f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:34.131389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe3f48eee8e4e81870753da5edae61a', 'request_id': 'req-8544c221-358c-48e6-898e-cbb5bc05f05f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:34.157097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] The token cache is being invalidate because project bbe3f48eee8e4e81870753da5edae61a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:43:34.157310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:34.158334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:34.160875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1294/2622] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:43:33 2022] DELETE /identity/v3/projects/bbe3f48eee8e4e81870753da5edae61a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:34.165329 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668ec0a-0ceb-4589-bab3-df3ff9a6071f service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:34.182772 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668ec0a-0ceb-4589-bab3-df3ff9a6071f service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:34.261120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938ec11-7de7-4112-94b5-3755173d812e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:34.261120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ec11-7de7-4112-94b5-3755173d812e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:34.261120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ec11-7de7-4112-94b5-3755173d812e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:34.262141 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ec11-7de7-4112-94b5-3755173d812e admin admin] PATH_INFO: `/v3/users/6067b880c0874a8294160e802a8552cf` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:34.264927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938ec11-7de7-4112-94b5-3755173d812e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6067b880c0874a8294160e802a8552cf)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:34.267172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938ec11-7de7-4112-94b5-3755173d812e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:34.495160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1938ec11-7de7-4112-94b5-3755173d812e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6067b880c0874a8294160e802a8552cf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:34.621520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1938ec11-7de7-4112-94b5-3755173d812e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6067b880c0874a8294160e802a8552cf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:34.636800 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1938ec11-7de7-4112-94b5-3755173d812e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6067b880c0874a8294160e802a8552cf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:34.644760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1938ec11-7de7-4112-94b5-3755173d812e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6067b880c0874a8294160e802a8552cf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:34.645524 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1938ec11-7de7-4112-94b5-3755173d812e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6067b880c0874a8294160e802a8552cf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:34.652601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1329/2623] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:43:34 2022] DELETE /identity/v3/users/6067b880c0874a8294160e802a8552cf => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:35.244723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:35.244723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:35.263039 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544c221-358c-48e6-898e-cbb5bc05f05f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:35.367723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:35.368709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:35.369056 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:35.369402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] PATH_INFO: `/v3/projects/9ec172047d694c48a54021b00aa153ea` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:35.373119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec172047d694c48a54021b00aa153ea)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:35.375160 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:35.404388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:35.515373 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec172047d694c48a54021b00aa153ea', 'request_id': 'req-aa2ba740-9863-467a-bda1-22eb271d06d5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:35.523241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec172047d694c48a54021b00aa153ea', 'request_id': 'req-aa2ba740-9863-467a-bda1-22eb271d06d5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:35.593980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec172047d694c48a54021b00aa153ea', 'request_id': 'req-aa2ba740-9863-467a-bda1-22eb271d06d5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:35.607040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] The token cache is being invalidate because project 9ec172047d694c48a54021b00aa153ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:43:35.607333 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:35.607999 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-aa2ba740-9863-467a-bda1-22eb271d06d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:35.609572 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1295/2624] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:43:35 2022] DELETE /identity/v3/projects/9ec172047d694c48a54021b00aa153ea => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:35.614744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5919e-5eba-4ad3-91b9-cbf034ce23ea None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:35.615262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5919e-5eba-4ad3-91b9-cbf034ce23ea None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:35.615364 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5919e-5eba-4ad3-91b9-cbf034ce23ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:35.636602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-45e5919e-5eba-4ad3-91b9-cbf034ce23ea None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:35.637819 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-45e5919e-5eba-4ad3-91b9-cbf034ce23ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:35.708778 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1330/2625] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:35.716737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec7739-c689-42ee-b7ee-4b634e16962d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:35.717021 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec7739-c689-42ee-b7ee-4b634e16962d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:35.717292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec7739-c689-42ee-b7ee-4b634e16962d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:35.738980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ddec7739-c689-42ee-b7ee-4b634e16962d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:35.740265 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ddec7739-c689-42ee-b7ee-4b634e16962d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:35.749255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1296/2626] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:35.753675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5919e-5eba-4ad3-91b9-cbf034ce23ea None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:35.765129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5919e-5eba-4ad3-91b9-cbf034ce23ea None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:35.772888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e71d6b-bccf-4797-b302-6c15c37cdab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:35.773674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e71d6b-bccf-4797-b302-6c15c37cdab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:35.774243 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e71d6b-bccf-4797-b302-6c15c37cdab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:35.774618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e71d6b-bccf-4797-b302-6c15c37cdab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:35.775414 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e71d6b-bccf-4797-b302-6c15c37cdab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:43:35.775825 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e71d6b-bccf-4797-b302-6c15c37cdab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:35.777795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e71d6b-bccf-4797-b302-6c15c37cdab1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:35.795610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:35.804065 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1331/2627] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:43:35.808126 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec7739-c689-42ee-b7ee-4b634e16962d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:35.822528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec7739-c689-42ee-b7ee-4b634e16962d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:35.838985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344ce9a-8b9b-4b2d-9091-6d7c6b36e47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:35.839606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344ce9a-8b9b-4b2d-9091-6d7c6b36e47d admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:35.839885 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344ce9a-8b9b-4b2d-9091-6d7c6b36e47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:35.840189 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344ce9a-8b9b-4b2d-9091-6d7c6b36e47d admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:35.840815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344ce9a-8b9b-4b2d-9091-6d7c6b36e47d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:35.842710 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344ce9a-8b9b-4b2d-9091-6d7c6b36e47d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:35.968396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1297/2628] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:43:35 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:43:35.979016 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e71d6b-bccf-4797-b302-6c15c37cdab1 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:36.001407 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e71d6b-bccf-4797-b302-6c15c37cdab1 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:36.097114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551fac5-2787-41f4-9cde-bdda710be71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:36.097718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551fac5-2787-41f4-9cde-bdda710be71a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:36.098000 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551fac5-2787-41f4-9cde-bdda710be71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:36.098276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551fac5-2787-41f4-9cde-bdda710be71a admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:36.098965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551fac5-2787-41f4-9cde-bdda710be71a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:36.101683 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551fac5-2787-41f4-9cde-bdda710be71a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:36.183629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1332/2629] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:43:35 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:43:36.187791 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344ce9a-8b9b-4b2d-9091-6d7c6b36e47d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:36.199520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344ce9a-8b9b-4b2d-9091-6d7c6b36e47d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:36.204956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a8547-0a23-4b92-8bcf-30d9af787db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:36.205570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a8547-0a23-4b92-8bcf-30d9af787db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:36.205846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a8547-0a23-4b92-8bcf-30d9af787db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:36.206130 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a8547-0a23-4b92-8bcf-30d9af787db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:36.206772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a8547-0a23-4b92-8bcf-30d9af787db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:36.208803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a8547-0a23-4b92-8bcf-30d9af787db1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:36.224112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1298/2630] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:36.228530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551fac5-2787-41f4-9cde-bdda710be71a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:36.239707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551fac5-2787-41f4-9cde-bdda710be71a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:36.244691 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74afacf-b4f7-484d-ae58-ae4d999b8b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:36.245359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74afacf-b4f7-484d-ae58-ae4d999b8b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:36.245637 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74afacf-b4f7-484d-ae58-ae4d999b8b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:36.245911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74afacf-b4f7-484d-ae58-ae4d999b8b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:36.246574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74afacf-b4f7-484d-ae58-ae4d999b8b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:36.248221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74afacf-b4f7-484d-ae58-ae4d999b8b68 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:36.262060 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1333/2631] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:36.264889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a8547-0a23-4b92-8bcf-30d9af787db1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:36.278352 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a8547-0a23-4b92-8bcf-30d9af787db1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:36.283228 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933aad7-3396-492c-b7e5-1036c4a65992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:36.284192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933aad7-3396-492c-b7e5-1036c4a65992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:36.284484 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933aad7-3396-492c-b7e5-1036c4a65992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:36.284763 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933aad7-3396-492c-b7e5-1036c4a65992 admin admin] PATH_INFO: `/v3/projects/9ee2e9cac93a41b0b0ae2007c620dbf8/users/b5fef051df4148308fe1531945d84fbf/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:36.301992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933aad7-3396-492c-b7e5-1036c4a65992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee2e9cac93a41b0b0ae2007c620dbf8, user_id=b5fef051df4148308fe1531945d84fbf, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:36.305154 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933aad7-3396-492c-b7e5-1036c4a65992 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:36.405545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1299/2632] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:43:36 2022] PUT /identity/v3/projects/9ee2e9cac93a41b0b0ae2007c620dbf8/users/b5fef051df4148308fe1531945d84fbf/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:43:36.416236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa64d7-6911-44fa-ac2f-ce36984b5120 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:36.416557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa64d7-6911-44fa-ac2f-ce36984b5120 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:36.416803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa64d7-6911-44fa-ac2f-ce36984b5120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:36.447462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f9aa64d7-6911-44fa-ac2f-ce36984b5120 None None] MFA Rules not processed for user `b5fef051df4148308fe1531945d84fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:36.448827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f9aa64d7-6911-44fa-ac2f-ce36984b5120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:36.524169 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1334/2633] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:36.532903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933aad7-3396-492c-b7e5-1036c4a65992 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:36.534958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e933aad7-3396-492c-b7e5-1036c4a65992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:36.625529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933aad7-3396-492c-b7e5-1036c4a65992 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:36.629506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf209c-5914-4c24-8ed5-74417513c01a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:36.631159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf209c-5914-4c24-8ed5-74417513c01a service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:36.631159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf209c-5914-4c24-8ed5-74417513c01a service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:36.631159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf209c-5914-4c24-8ed5-74417513c01a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:36.641133 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf209c-5914-4c24-8ed5-74417513c01a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:36.642574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf209c-5914-4c24-8ed5-74417513c01a service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:36.654292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1300/2634] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:43:36.676040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa64d7-6911-44fa-ac2f-ce36984b5120 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:36.790387 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa64d7-6911-44fa-ac2f-ce36984b5120 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:36.814649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:36.815468 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:36.815888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:36.816251 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] PATH_INFO: `/v3/users/b5fef051df4148308fe1531945d84fbf` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:36.817693 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5fef051df4148308fe1531945d84fbf)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:36.819888 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:36.828136 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:37.057355 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5fef051df4148308fe1531945d84fbf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:37.132690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5fef051df4148308fe1531945d84fbf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:37.157642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5fef051df4148308fe1531945d84fbf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:37.170570 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5fef051df4148308fe1531945d84fbf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:37.171478 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5fef051df4148308fe1531945d84fbf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:37.188310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1335/2635] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:43:36 2022] DELETE /identity/v3/users/b5fef051df4148308fe1531945d84fbf => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:37.192971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da21b-a3d4-4f4a-99a3-692ecc6c2f13 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:37.193699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da21b-a3d4-4f4a-99a3-692ecc6c2f13 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:37.194235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da21b-a3d4-4f4a-99a3-692ecc6c2f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:37.231120 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-214da21b-a3d4-4f4a-99a3-692ecc6c2f13 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:37.233144 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-214da21b-a3d4-4f4a-99a3-692ecc6c2f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:37.291640 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:37.320135 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1301/2636] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:43:37.331392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:37.337947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:37.416189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda532bd-8a66-4129-aacd-231d8275ff76 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:37.423095 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df1500-0262-4193-8225-72a857d18b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:37.423095 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df1500-0262-4193-8225-72a857d18b60 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:37.423095 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df1500-0262-4193-8225-72a857d18b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:37.423095 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df1500-0262-4193-8225-72a857d18b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:37.434286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df1500-0262-4193-8225-72a857d18b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:37.436339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df1500-0262-4193-8225-72a857d18b60 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:37.449903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1336/2637] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:38.417413 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214da21b-a3d4-4f4a-99a3-692ecc6c2f13 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:38.422553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-214da21b-a3d4-4f4a-99a3-692ecc6c2f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:38.449125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214da21b-a3d4-4f4a-99a3-692ecc6c2f13 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:38.467947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:38.468713 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:38.469065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:38.469243 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] PATH_INFO: `/v3/projects/9ee2e9cac93a41b0b0ae2007c620dbf8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:38.471520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee2e9cac93a41b0b0ae2007c620dbf8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:38.473532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:38.617628 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee2e9cac93a41b0b0ae2007c620dbf8', 'request_id': 'req-89fc5e97-4304-4dcb-875a-bde90c031d04'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:38.631052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee2e9cac93a41b0b0ae2007c620dbf8', 'request_id': 'req-89fc5e97-4304-4dcb-875a-bde90c031d04'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:38.703623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee2e9cac93a41b0b0ae2007c620dbf8', 'request_id': 'req-89fc5e97-4304-4dcb-875a-bde90c031d04'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:38.719053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] The token cache is being invalidate because project 9ee2e9cac93a41b0b0ae2007c620dbf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:43:38.719415 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:38.720054 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-89fc5e97-4304-4dcb-875a-bde90c031d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:38.721774 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1302/2638] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:43:38 2022] DELETE /identity/v3/projects/9ee2e9cac93a41b0b0ae2007c620dbf8 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:38.728122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fe08b-bd5c-4e6a-ad2a-7f96bf619e02 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:38.732741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fe08b-bd5c-4e6a-ad2a-7f96bf619e02 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:38.732741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fe08b-bd5c-4e6a-ad2a-7f96bf619e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:38.750581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d58fe08b-bd5c-4e6a-ad2a-7f96bf619e02 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:38.751894 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d58fe08b-bd5c-4e6a-ad2a-7f96bf619e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:38.836436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68cadd-582e-4fe0-b79a-db87c3e0e755 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:38.836722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68cadd-582e-4fe0-b79a-db87c3e0e755 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:38.836998 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68cadd-582e-4fe0-b79a-db87c3e0e755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:38.841286 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1337/2639] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:38.859887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7b68cadd-582e-4fe0-b79a-db87c3e0e755 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:38.861169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7b68cadd-582e-4fe0-b79a-db87c3e0e755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:38.869965 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1303/2640] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:38.874380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fe08b-bd5c-4e6a-ad2a-7f96bf619e02 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:38.884804 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fe08b-bd5c-4e6a-ad2a-7f96bf619e02 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:38.889780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdecad-719a-4613-bb84-fb674dc4817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:38.893170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdecad-719a-4613-bb84-fb674dc4817e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:38.893170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdecad-719a-4613-bb84-fb674dc4817e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:38.893170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdecad-719a-4613-bb84-fb674dc4817e admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:38.893170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdecad-719a-4613-bb84-fb674dc4817e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:43:38.893170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdecad-719a-4613-bb84-fb674dc4817e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:38.893687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdecad-719a-4613-bb84-fb674dc4817e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:38.909291 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1338/2641] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:43:38.911377 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68cadd-582e-4fe0-b79a-db87c3e0e755 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:38.922452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68cadd-582e-4fe0-b79a-db87c3e0e755 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:38.927564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f27b59-6708-4b1c-8384-4d4ccbbefe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:38.942911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f27b59-6708-4b1c-8384-4d4ccbbefe1d admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:38.942911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f27b59-6708-4b1c-8384-4d4ccbbefe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:38.942911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f27b59-6708-4b1c-8384-4d4ccbbefe1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:38.942911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f27b59-6708-4b1c-8384-4d4ccbbefe1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:38.942911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f27b59-6708-4b1c-8384-4d4ccbbefe1d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:39.063004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1304/2642] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:43:38 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:43:39.067036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdecad-719a-4613-bb84-fb674dc4817e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:39.077331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdecad-719a-4613-bb84-fb674dc4817e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:39.089439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:39.142129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019a7e9-0b18-432b-87a6-c03fae2b932a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:39.142762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019a7e9-0b18-432b-87a6-c03fae2b932a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:39.143420 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019a7e9-0b18-432b-87a6-c03fae2b932a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:39.143733 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019a7e9-0b18-432b-87a6-c03fae2b932a admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:39.144404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019a7e9-0b18-432b-87a6-c03fae2b932a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:39.146377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019a7e9-0b18-432b-87a6-c03fae2b932a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:39.255871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1339/2643] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:43:39 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:43:39.260340 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f27b59-6708-4b1c-8384-4d4ccbbefe1d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:39.270842 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f27b59-6708-4b1c-8384-4d4ccbbefe1d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:39.275974 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4affa-5283-41a4-af92-d3bcf58bd4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:39.276588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4affa-5283-41a4-af92-d3bcf58bd4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:39.276862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4affa-5283-41a4-af92-d3bcf58bd4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:39.277131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4affa-5283-41a4-af92-d3bcf58bd4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:39.277772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4affa-5283-41a4-af92-d3bcf58bd4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:39.279335 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4affa-5283-41a4-af92-d3bcf58bd4f2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:39.288949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1305/2644] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:39.292746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019a7e9-0b18-432b-87a6-c03fae2b932a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:39.302532 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019a7e9-0b18-432b-87a6-c03fae2b932a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:39.307105 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473b300-cc87-4bc9-9b8e-687afabefa31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:39.307688 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473b300-cc87-4bc9-9b8e-687afabefa31 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:39.307966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473b300-cc87-4bc9-9b8e-687afabefa31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:39.308264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473b300-cc87-4bc9-9b8e-687afabefa31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:39.308900 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473b300-cc87-4bc9-9b8e-687afabefa31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:39.310479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473b300-cc87-4bc9-9b8e-687afabefa31 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:39.320744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1340/2645] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:43:39.324970 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4affa-5283-41a4-af92-d3bcf58bd4f2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:39.335051 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4affa-5283-41a4-af92-d3bcf58bd4f2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:39.340508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753e3b3-00f7-4df5-a34e-16ffec3f752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:39.341177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753e3b3-00f7-4df5-a34e-16ffec3f752c admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:39.341455 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753e3b3-00f7-4df5-a34e-16ffec3f752c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:39.341727 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753e3b3-00f7-4df5-a34e-16ffec3f752c admin admin] PATH_INFO: `/v3/projects/c46659202971413d893529941b799697/users/e05ed9257f8b446abe31525580c093ee/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:39.358680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753e3b3-00f7-4df5-a34e-16ffec3f752c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46659202971413d893529941b799697, user_id=e05ed9257f8b446abe31525580c093ee, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:39.361410 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753e3b3-00f7-4df5-a34e-16ffec3f752c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:39.421743 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1306/2646] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:43:39 2022] PUT /identity/v3/projects/c46659202971413d893529941b799697/users/e05ed9257f8b446abe31525580c093ee/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:39.432112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf0384-56b7-40d9-a2f7-61ebc6aaa617 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:39.433895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf0384-56b7-40d9-a2f7-61ebc6aaa617 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:39.433895 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf0384-56b7-40d9-a2f7-61ebc6aaa617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:39.461501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1bcf0384-56b7-40d9-a2f7-61ebc6aaa617 None None] MFA Rules not processed for user `e05ed9257f8b446abe31525580c093ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:39.462732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1bcf0384-56b7-40d9-a2f7-61ebc6aaa617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:39.526475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1341/2647] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:39.537332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753e3b3-00f7-4df5-a34e-16ffec3f752c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:39.539077 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a753e3b3-00f7-4df5-a34e-16ffec3f752c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:39.595506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753e3b3-00f7-4df5-a34e-16ffec3f752c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:39.599667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9333d41-a7eb-4528-898c-00c5c2bc2dd8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb2129c006f4ddeba7d128a8d43d5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:39.600457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9333d41-a7eb-4528-898c-00c5c2bc2dd8 service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:39.606080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9333d41-a7eb-4528-898c-00c5c2bc2dd8 service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:39.606080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9333d41-a7eb-4528-898c-00c5c2bc2dd8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:39.610875 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9333d41-a7eb-4528-898c-00c5c2bc2dd8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:39.612354 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9333d41-a7eb-4528-898c-00c5c2bc2dd8 service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:39.622642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1307/2648] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:42.498731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf0384-56b7-40d9-a2f7-61ebc6aaa617 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:42.512633 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1bcf0384-56b7-40d9-a2f7-61ebc6aaa617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:42.602719 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf0384-56b7-40d9-a2f7-61ebc6aaa617 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:42.609405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef1142-6faf-407c-8bcb-710797e3554f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:42.610028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef1142-6faf-407c-8bcb-710797e3554f service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:42.610314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef1142-6faf-407c-8bcb-710797e3554f service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:42.610802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef1142-6faf-407c-8bcb-710797e3554f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:42.612472 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c5ef1142-6faf-407c-8bcb-710797e3554f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:42.628346 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef1142-6faf-407c-8bcb-710797e3554f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:42.629987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef1142-6faf-407c-8bcb-710797e3554f service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:42.643652 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1342/2649] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:43:45.619966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ffa25-aee6-4003-ab5d-a3995e8089bf None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:45.620606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ffa25-aee6-4003-ab5d-a3995e8089bf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:45.620726 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ffa25-aee6-4003-ab5d-a3995e8089bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:45.639722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a56ffa25-aee6-4003-ab5d-a3995e8089bf None None] MFA Rules not processed for user `394c67bcc3864bb2aca2f4faec834029`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:45.640961 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a56ffa25-aee6-4003-ab5d-a3995e8089bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:45.644903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:45.698437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1308/2650] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:46.036414 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef1142-6faf-407c-8bcb-710797e3554f service nova] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:46.037869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c5ef1142-6faf-407c-8bcb-710797e3554f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:46.050625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:46.103381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef1142-6faf-407c-8bcb-710797e3554f service nova] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:46.107334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352a88d-b228-460b-ac41-a7f0db889d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:46.107924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a88d-b228-460b-ac41-a7f0db889d75 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:46.108244 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a88d-b228-460b-ac41-a7f0db889d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:46.108518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a88d-b228-460b-ac41-a7f0db889d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:46.110308 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9352a88d-b228-460b-ac41-a7f0db889d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:46.120821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352a88d-b228-460b-ac41-a7f0db889d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:46.122386 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352a88d-b228-460b-ac41-a7f0db889d75 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:46.131150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1343/2651] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:47.608073 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cac55-55ac-4e9b-9875-c0a36e043266 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:47.608650 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cac55-55ac-4e9b-9875-c0a36e043266 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:47.608650 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cac55-55ac-4e9b-9875-c0a36e043266 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:47.610148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1309/2652] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:43:47.618955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc4633-c45f-4a86-b88f-006df2c243f9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:47.619666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc4633-c45f-4a86-b88f-006df2c243f9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:47.619666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc4633-c45f-4a86-b88f-006df2c243f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:47.641361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-23bc4633-c45f-4a86-b88f-006df2c243f9 None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:47.642751 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-23bc4633-c45f-4a86-b88f-006df2c243f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:47.657779 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1344/2653] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:43:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:47.670990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cac55-55ac-4e9b-9875-c0a36e043266 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:47.672904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-334cac55-55ac-4e9b-9875-c0a36e043266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:47.694248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cac55-55ac-4e9b-9875-c0a36e043266 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:47.699005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b9f9b-c697-4911-99c7-326f11bead3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb2129c006f4ddeba7d128a8d43d5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:47.699934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b9f9b-c697-4911-99c7-326f11bead3c service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:47.700515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b9f9b-c697-4911-99c7-326f11bead3c service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:47.700889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b9f9b-c697-4911-99c7-326f11bead3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:47.712735 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b9f9b-c697-4911-99c7-326f11bead3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:47.714293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b9f9b-c697-4911-99c7-326f11bead3c service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:47.728623 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1310/2654] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:47.945938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9b9a5-cb92-4927-8aa3-3b403a3d1272 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:47.948008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9b9a5-cb92-4927-8aa3-3b403a3d1272 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:47.948008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9b9a5-cb92-4927-8aa3-3b403a3d1272 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:47.949128 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1345/2655] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:43:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:43:47.955887 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d9c5-b213-4877-bb12-d5c59ff4c4f7 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:47.956343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d9c5-b213-4877-bb12-d5c59ff4c4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:47.956967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d9c5-b213-4877-bb12-d5c59ff4c4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:47.991804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f891d9c5-b213-4877-bb12-d5c59ff4c4f7 None None] MFA Rules not processed for user `baffd4f6a8ad4a6e91d8d9c61cc1dce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:47.993534 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f891d9c5-b213-4877-bb12-d5c59ff4c4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:48.005968 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1311/2656] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:43:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:48.025206 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9b9a5-cb92-4927-8aa3-3b403a3d1272 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:48.027135 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f8e9b9a5-cb92-4927-8aa3-3b403a3d1272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:48.043969 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9b9a5-cb92-4927-8aa3-3b403a3d1272 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:48.052582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585bb1d-5a51-404e-96e8-11cfcb2d92fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:48.053334 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585bb1d-5a51-404e-96e8-11cfcb2d92fc service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:48.053715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585bb1d-5a51-404e-96e8-11cfcb2d92fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:48.054043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585bb1d-5a51-404e-96e8-11cfcb2d92fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:48.062253 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585bb1d-5a51-404e-96e8-11cfcb2d92fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:48.063753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585bb1d-5a51-404e-96e8-11cfcb2d92fc service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:48.072166 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1346/2657] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:48.174724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e5a33-ef47-498b-913c-06e096603a61 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:48.179680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e5a33-ef47-498b-913c-06e096603a61 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:48.179680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e5a33-ef47-498b-913c-06e096603a61 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:48.179680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1312/2658] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 22:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 22:43:48.183343 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af7518-6b6b-4479-86a8-25b989ab5dcb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:48.183729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af7518-6b6b-4479-86a8-25b989ab5dcb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:48.184270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af7518-6b6b-4479-86a8-25b989ab5dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:48.208656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-85af7518-6b6b-4479-86a8-25b989ab5dcb None None] MFA Rules not processed for user `8effbd0670ef41f59b46feb9f0997554`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:48.210078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-85af7518-6b6b-4479-86a8-25b989ab5dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:48.220485 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1347/2659] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 22:43:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:48.235736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e5a33-ef47-498b-913c-06e096603a61 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:48.250844 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e5a33-ef47-498b-913c-06e096603a61 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:48.255139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56a597-644a-4260-ad20-0f1706cbc7c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:48.255771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56a597-644a-4260-ad20-0f1706cbc7c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:48.259986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56a597-644a-4260-ad20-0f1706cbc7c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:48.259986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56a597-644a-4260-ad20-0f1706cbc7c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:48.269361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56a597-644a-4260-ad20-0f1706cbc7c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:48.271034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56a597-644a-4260-ad20-0f1706cbc7c3 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:48.286772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1313/2660] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:51.161424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d9f7b-5f12-47f8-b16a-3d07dbef05d5 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:51.161957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d9f7b-5f12-47f8-b16a-3d07dbef05d5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:51.161957 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d9f7b-5f12-47f8-b16a-3d07dbef05d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:51.183329 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-823d9f7b-5f12-47f8-b16a-3d07dbef05d5 None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:51.184729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-823d9f7b-5f12-47f8-b16a-3d07dbef05d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:51.250162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1348/2661] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:43:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:51.263179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56a597-644a-4260-ad20-0f1706cbc7c3 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:51.264658 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2b56a597-644a-4260-ad20-0f1706cbc7c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:51.340070 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56a597-644a-4260-ad20-0f1706cbc7c3 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:51.345404 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a3289-1da6-4516-9bc5-d7f8f57720ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:51.346021 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a3289-1da6-4516-9bc5-d7f8f57720ef service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:51.352893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a3289-1da6-4516-9bc5-d7f8f57720ef service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:51.352893 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a3289-1da6-4516-9bc5-d7f8f57720ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:51.355853 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a3289-1da6-4516-9bc5-d7f8f57720ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:51.357532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a3289-1da6-4516-9bc5-d7f8f57720ef service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:51.365947 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1314/2662] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:43:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:43:58.333909 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d9f7b-5f12-47f8-b16a-3d07dbef05d5 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:58.431865 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d9f7b-5f12-47f8-b16a-3d07dbef05d5 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:58.436535 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:58.437170 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:58.437475 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:58.437746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] PATH_INFO: `/v3/users/e05ed9257f8b446abe31525580c093ee` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:58.438885 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e05ed9257f8b446abe31525580c093ee)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:58.440843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:58.840140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:58.860789 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e05ed9257f8b446abe31525580c093ee'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:58.946306 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e05ed9257f8b446abe31525580c093ee'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:58.959519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e05ed9257f8b446abe31525580c093ee'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:58.967296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e05ed9257f8b446abe31525580c093ee'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:58.968072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e05ed9257f8b446abe31525580c093ee'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:58.985134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1349/2663] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:43:58 2022] DELETE /identity/v3/users/e05ed9257f8b446abe31525580c093ee => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:43:58.988971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1663e-537f-4175-980b-e7c5ce0cb66d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:58.988971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1663e-537f-4175-980b-e7c5ce0cb66d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:58.988971 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1663e-537f-4175-980b-e7c5ce0cb66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:59.006989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3da1663e-537f-4175-980b-e7c5ce0cb66d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:43:59.008382 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3da1663e-537f-4175-980b-e7c5ce0cb66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:59.073712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1315/2664] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:43:59.083504 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:59.085762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:59.147042 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed8663-2c78-45b3-8c8f-ad9df7e5bef3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:59.151318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec0184-7cf9-4cf3-b0e4-c7a18ae42cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:59.151923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0184-7cf9-4cf3-b0e4-c7a18ae42cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:59.152230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0184-7cf9-4cf3-b0e4-c7a18ae42cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:59.152505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0184-7cf9-4cf3-b0e4-c7a18ae42cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:59.161409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec0184-7cf9-4cf3-b0e4-c7a18ae42cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:59.167734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec0184-7cf9-4cf3-b0e4-c7a18ae42cc5 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:59.172842 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1350/2665] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:43:59.770613 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1663e-537f-4175-980b-e7c5ce0cb66d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:43:59.772924 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3da1663e-537f-4175-980b-e7c5ce0cb66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:43:59.786300 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1663e-537f-4175-980b-e7c5ce0cb66d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:43:59.791241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:43:59.791914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:43:59.792232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:43:59.792511 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] PATH_INFO: `/v3/projects/c46659202971413d893529941b799697` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:43:59.793824 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46659202971413d893529941b799697)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:43:59.795771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:43:59.803301 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:43:59.926002 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46659202971413d893529941b799697', 'request_id': 'req-c7e801f7-e063-4ee9-b24d-320b4155c701'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:43:59.934357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46659202971413d893529941b799697', 'request_id': 'req-c7e801f7-e063-4ee9-b24d-320b4155c701'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:44:00.039533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46659202971413d893529941b799697', 'request_id': 'req-c7e801f7-e063-4ee9-b24d-320b4155c701'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:44:00.052978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] The token cache is being invalidate because project c46659202971413d893529941b799697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:44:00.053850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:44:00.054643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c7e801f7-e063-4ee9-b24d-320b4155c701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:44:00.056177 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1316/2666] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:43:59 2022] DELETE /identity/v3/projects/c46659202971413d893529941b799697 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:44:00.061728 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a97ab-0d17-4eca-8cd4-034d5180ed8d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:00.062278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a97ab-0d17-4eca-8cd4-034d5180ed8d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:00.062549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a97ab-0d17-4eca-8cd4-034d5180ed8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:00.083018 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-646a97ab-0d17-4eca-8cd4-034d5180ed8d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:44:00.084260 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-646a97ab-0d17-4eca-8cd4-034d5180ed8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:00.156228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1351/2667] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:44:00.163462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89638-bcd5-4b1d-8dee-a86bcf549f1a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:00.164236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89638-bcd5-4b1d-8dee-a86bcf549f1a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:00.164754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89638-bcd5-4b1d-8dee-a86bcf549f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:00.193162 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6ec89638-bcd5-4b1d-8dee-a86bcf549f1a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:44:00.195531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6ec89638-bcd5-4b1d-8dee-a86bcf549f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:00.205093 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1317/2668] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:44:00.209347 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a97ab-0d17-4eca-8cd4-034d5180ed8d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:00.219554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a97ab-0d17-4eca-8cd4-034d5180ed8d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:00.224863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b7b9b-cfa6-4160-a598-406b49ae5f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:00.227922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b7b9b-cfa6-4160-a598-406b49ae5f4b admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:00.227922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b7b9b-cfa6-4160-a598-406b49ae5f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:00.227922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b7b9b-cfa6-4160-a598-406b49ae5f4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:00.227922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b7b9b-cfa6-4160-a598-406b49ae5f4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:44:00.227922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b7b9b-cfa6-4160-a598-406b49ae5f4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:00.228872 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b7b9b-cfa6-4160-a598-406b49ae5f4b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:00.241795 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1352/2669] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:44:00.246213 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec89638-bcd5-4b1d-8dee-a86bcf549f1a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:00.255396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec89638-bcd5-4b1d-8dee-a86bcf549f1a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:00.260732 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8690a3-eda0-4426-9562-292018779d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:00.261581 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8690a3-eda0-4426-9562-292018779d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:00.261861 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8690a3-eda0-4426-9562-292018779d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:00.262134 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8690a3-eda0-4426-9562-292018779d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:00.262751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8690a3-eda0-4426-9562-292018779d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:00.264670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8690a3-eda0-4426-9562-292018779d2a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:00.391015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1318/2670] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:44:00 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:44:00.395401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b7b9b-cfa6-4160-a598-406b49ae5f4b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:00.405570 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b7b9b-cfa6-4160-a598-406b49ae5f4b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:00.446139 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:44:00.464378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca4191-1e99-43f3-aca1-fc9346464d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:00.464982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca4191-1e99-43f3-aca1-fc9346464d45 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:00.465292 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca4191-1e99-43f3-aca1-fc9346464d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:00.465567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca4191-1e99-43f3-aca1-fc9346464d45 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:00.466203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca4191-1e99-43f3-aca1-fc9346464d45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:00.468150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca4191-1e99-43f3-aca1-fc9346464d45 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:00.541440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1353/2671] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:44:00 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:44:00.545574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8690a3-eda0-4426-9562-292018779d2a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:00.556400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8690a3-eda0-4426-9562-292018779d2a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:00.561171 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681778e5-6d46-4712-bad2-cfdc47396782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:00.561772 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681778e5-6d46-4712-bad2-cfdc47396782 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:00.562054 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681778e5-6d46-4712-bad2-cfdc47396782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:00.562331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681778e5-6d46-4712-bad2-cfdc47396782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:00.562980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681778e5-6d46-4712-bad2-cfdc47396782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:00.564680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681778e5-6d46-4712-bad2-cfdc47396782 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:00.575334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1319/2672] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:44:00.579285 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca4191-1e99-43f3-aca1-fc9346464d45 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:00.590495 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca4191-1e99-43f3-aca1-fc9346464d45 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:00.595256 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217a2e7-bfa2-4544-bf32-ae6ff973df3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:00.595855 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a2e7-bfa2-4544-bf32-ae6ff973df3e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:00.596177 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a2e7-bfa2-4544-bf32-ae6ff973df3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:00.596455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a2e7-bfa2-4544-bf32-ae6ff973df3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:00.597137 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217a2e7-bfa2-4544-bf32-ae6ff973df3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:00.598735 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217a2e7-bfa2-4544-bf32-ae6ff973df3e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:00.608741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1354/2673] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:44:00.612699 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681778e5-6d46-4712-bad2-cfdc47396782 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:00.622960 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681778e5-6d46-4712-bad2-cfdc47396782 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:00.627903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5735afe-947c-4e92-8edf-9287d3ada80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:00.631836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5735afe-947c-4e92-8edf-9287d3ada80b admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:00.631836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5735afe-947c-4e92-8edf-9287d3ada80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:00.631836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5735afe-947c-4e92-8edf-9287d3ada80b admin admin] PATH_INFO: `/v3/projects/2648957a96504832a5693026f9d99c58/users/269193a67b504842b3858022b378ae2a/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:00.644788 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5735afe-947c-4e92-8edf-9287d3ada80b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2648957a96504832a5693026f9d99c58, user_id=269193a67b504842b3858022b378ae2a, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:00.648046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5735afe-947c-4e92-8edf-9287d3ada80b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:00.708179 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1320/2674] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:44:00 2022] PUT /identity/v3/projects/2648957a96504832a5693026f9d99c58/users/269193a67b504842b3858022b378ae2a/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:44:00.713718 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8397ec-9acb-476f-a273-b732960192f6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:00.714034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8397ec-9acb-476f-a273-b732960192f6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:00.714314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8397ec-9acb-476f-a273-b732960192f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:00.733886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ec8397ec-9acb-476f-a273-b732960192f6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:44:00.741852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ec8397ec-9acb-476f-a273-b732960192f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:00.815300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1355/2675] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:44:00.825187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5735afe-947c-4e92-8edf-9287d3ada80b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:00.826911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c5735afe-947c-4e92-8edf-9287d3ada80b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:00.888950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5735afe-947c-4e92-8edf-9287d3ada80b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:00.892654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8ebbd-39c5-4e9e-9bf4-13463b85a465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:00.893293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8ebbd-39c5-4e9e-9bf4-13463b85a465 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:00.893577 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8ebbd-39c5-4e9e-9bf4-13463b85a465 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:00.893851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8ebbd-39c5-4e9e-9bf4-13463b85a465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:00.904585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8ebbd-39c5-4e9e-9bf4-13463b85a465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:00.905955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8ebbd-39c5-4e9e-9bf4-13463b85a465 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:00.916982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1321/2676] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:44:02.339689 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75c70-24d7-4955-85c9-9836ead5ef83 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:02.340317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75c70-24d7-4955-85c9-9836ead5ef83 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:02.340317 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75c70-24d7-4955-85c9-9836ead5ef83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:02.368083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-36f75c70-24d7-4955-85c9-9836ead5ef83 None None] MFA Rules not processed for user `269193a67b504842b3858022b378ae2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:44:02.369415 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-36f75c70-24d7-4955-85c9-9836ead5ef83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:02.449428 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1356/2677] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:44:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:44:02.460235 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8ebbd-39c5-4e9e-9bf4-13463b85a465 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:02.461642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-caa8ebbd-39c5-4e9e-9bf4-13463b85a465 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:02.497420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:44:02.518952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8ebbd-39c5-4e9e-9bf4-13463b85a465 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:02.522597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2b520-c5ff-4549-bf28-e7e7c6dfa89d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:02.523201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2b520-c5ff-4549-bf28-e7e7c6dfa89d service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:02.523478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2b520-c5ff-4549-bf28-e7e7c6dfa89d service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:02.523748 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2b520-c5ff-4549-bf28-e7e7c6dfa89d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:02.532866 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2b520-c5ff-4549-bf28-e7e7c6dfa89d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:02.534355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2b520-c5ff-4549-bf28-e7e7c6dfa89d service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:02.544247 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1322/2678] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:44:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:44:04.249608 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da8472-fdc6-4079-a1c7-3ef3212734b7 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:04.250146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da8472-fdc6-4079-a1c7-3ef3212734b7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:04.250146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da8472-fdc6-4079-a1c7-3ef3212734b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:04.278120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-12da8472-fdc6-4079-a1c7-3ef3212734b7 None None] MFA Rules not processed for user `269193a67b504842b3858022b378ae2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:44:04.279543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-12da8472-fdc6-4079-a1c7-3ef3212734b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:04.286881 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1357/2679] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:44:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:44:06.433259 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd71a6-d4ca-4f4a-a64b-c0adc01816e4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:06.433790 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd71a6-d4ca-4f4a-a64b-c0adc01816e4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:06.433790 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd71a6-d4ca-4f4a-a64b-c0adc01816e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:06.456787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-59bd71a6-d4ca-4f4a-a64b-c0adc01816e4 None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:44:06.458431 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-59bd71a6-d4ca-4f4a-a64b-c0adc01816e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:06.558833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1323/2680] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 22:44:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:44:06.572254 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da8472-fdc6-4079-a1c7-3ef3212734b7 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:06.574164 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-12da8472-fdc6-4079-a1c7-3ef3212734b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:06.624102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:44:06.648533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da8472-fdc6-4079-a1c7-3ef3212734b7 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:06.654247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe83e2-0694-441d-89d0-30be75023fb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:06.654855 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe83e2-0694-441d-89d0-30be75023fb6 service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:06.655133 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe83e2-0694-441d-89d0-30be75023fb6 service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:06.655405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe83e2-0694-441d-89d0-30be75023fb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:06.664578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe83e2-0694-441d-89d0-30be75023fb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:06.666237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe83e2-0694-441d-89d0-30be75023fb6 service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:06.675240 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1358/2681] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:44:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:44:09.128509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd71a6-d4ca-4f4a-a64b-c0adc01816e4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:09.144326 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd71a6-d4ca-4f4a-a64b-c0adc01816e4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:09.148534 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e682a-e716-4f02-93f2-d3997a5bf645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:09.149252 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e682a-e716-4f02-93f2-d3997a5bf645 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:09.149629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e682a-e716-4f02-93f2-d3997a5bf645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:09.150381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e682a-e716-4f02-93f2-d3997a5bf645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:09.151800 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-771e682a-e716-4f02-93f2-d3997a5bf645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:09.223903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e682a-e716-4f02-93f2-d3997a5bf645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:09.226267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e682a-e716-4f02-93f2-d3997a5bf645 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:09.248556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1324/2682] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:44:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:44:16.696529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=13 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:44:19.234102 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=66 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:44:26.734341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ece40-d63f-4345-a963-e9c242173664 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:26.734808 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ece40-d63f-4345-a963-e9c242173664 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:26.734923 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ece40-d63f-4345-a963-e9c242173664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:26.754325 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-623ece40-d63f-4345-a963-e9c242173664 None None] MFA Rules not processed for user `269193a67b504842b3858022b378ae2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:44:26.755474 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-623ece40-d63f-4345-a963-e9c242173664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:26.763342 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1359/2683] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:44:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:44:36.749634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:44:49.394718 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d4d65-a996-4a0f-b07d-8e1977de21bc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:49.395200 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d4d65-a996-4a0f-b07d-8e1977de21bc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:49.395298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d4d65-a996-4a0f-b07d-8e1977de21bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:49.416414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-460d4d65-a996-4a0f-b07d-8e1977de21bc None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:44:49.417819 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-460d4d65-a996-4a0f-b07d-8e1977de21bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:49.425462 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1325/2684] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:44:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:44:49.446606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ece40-d63f-4345-a963-e9c242173664 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:49.458602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ece40-d63f-4345-a963-e9c242173664 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:49.464044 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426c998-5430-4d60-8e50-1dd692372330 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:49.464649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426c998-5430-4d60-8e50-1dd692372330 service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:49.465027 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426c998-5430-4d60-8e50-1dd692372330 service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:49.465352 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426c998-5430-4d60-8e50-1dd692372330 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:49.474571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426c998-5430-4d60-8e50-1dd692372330 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:49.476214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426c998-5430-4d60-8e50-1dd692372330 service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:49.485088 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1360/2685] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:44:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:44:57.606576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d4d65-a996-4a0f-b07d-8e1977de21bc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:44:57.608082 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-460d4d65-a996-4a0f-b07d-8e1977de21bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:57.621156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d4d65-a996-4a0f-b07d-8e1977de21bc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:44:57.625629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f4439-d4cb-477b-8905-f32e08212769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:44:57.626350 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f4439-d4cb-477b-8905-f32e08212769 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:44:57.626700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f4439-d4cb-477b-8905-f32e08212769 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:44:57.627049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f4439-d4cb-477b-8905-f32e08212769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:44:57.628881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-843f4439-d4cb-477b-8905-f32e08212769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:44:57.644607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f4439-d4cb-477b-8905-f32e08212769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:44:57.646181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f4439-d4cb-477b-8905-f32e08212769 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:44:57.657478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1326/2686] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:44:59.481798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:01.778839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426c998-5430-4d60-8e50-1dd692372330 service swift] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:01.780444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2426c998-5430-4d60-8e50-1dd692372330 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:01.891397 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426c998-5430-4d60-8e50-1dd692372330 service swift] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:01.904461 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b3175-70b3-4487-af08-53d48445d6cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:01.905092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b3175-70b3-4487-af08-53d48445d6cc service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:01.905374 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b3175-70b3-4487-af08-53d48445d6cc service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:01.905651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b3175-70b3-4487-af08-53d48445d6cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:01.907513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5c9b3175-70b3-4487-af08-53d48445d6cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:01.926588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f4439-d4cb-477b-8905-f32e08212769 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:01.928851 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-843f4439-d4cb-477b-8905-f32e08212769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:01.930221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b3175-70b3-4487-af08-53d48445d6cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:01.931822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b3175-70b3-4487-af08-53d48445d6cc service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:01.943983 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1361/2687] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:45:01.955497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f4439-d4cb-477b-8905-f32e08212769 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:01.962815 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167b374-759a-41b5-b669-a9ce62161f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:01.963454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167b374-759a-41b5-b669-a9ce62161f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:01.963734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167b374-759a-41b5-b669-a9ce62161f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:01.964321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167b374-759a-41b5-b669-a9ce62161f05 admin admin] PATH_INFO: `/v3/users/05ef5ec6afb84c4abc0cb4862f041dd1` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:01.965267 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167b374-759a-41b5-b669-a9ce62161f05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05ef5ec6afb84c4abc0cb4862f041dd1)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:01.967496 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167b374-759a-41b5-b669-a9ce62161f05 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:02.186374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1167b374-759a-41b5-b669-a9ce62161f05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05ef5ec6afb84c4abc0cb4862f041dd1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:02.261145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1167b374-759a-41b5-b669-a9ce62161f05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05ef5ec6afb84c4abc0cb4862f041dd1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:02.276975 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1167b374-759a-41b5-b669-a9ce62161f05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05ef5ec6afb84c4abc0cb4862f041dd1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:02.278029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1167b374-759a-41b5-b669-a9ce62161f05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05ef5ec6afb84c4abc0cb4862f041dd1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:02.278682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1167b374-759a-41b5-b669-a9ce62161f05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05ef5ec6afb84c4abc0cb4862f041dd1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:02.292933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1327/2688] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:45:01 2022] DELETE /identity/v3/users/05ef5ec6afb84c4abc0cb4862f041dd1 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:45:02.600634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b3175-70b3-4487-af08-53d48445d6cc service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:02.615618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b3175-70b3-4487-af08-53d48445d6cc service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:02.684296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:02.685811 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:02.686201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:02.686488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] PATH_INFO: `/v3/projects/3f7ea2694e7641088d957060a1001d86` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:02.687962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f7ea2694e7641088d957060a1001d86)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:02.690004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:02.824653 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f7ea2694e7641088d957060a1001d86', 'request_id': 'req-2d800aed-c18c-4f9d-8be1-ab5294d140f3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:02.831385 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f7ea2694e7641088d957060a1001d86', 'request_id': 'req-2d800aed-c18c-4f9d-8be1-ab5294d140f3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:02.893453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f7ea2694e7641088d957060a1001d86', 'request_id': 'req-2d800aed-c18c-4f9d-8be1-ab5294d140f3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:02.908984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] The token cache is being invalidate because project 3f7ea2694e7641088d957060a1001d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:45:02.909579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:02.910501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d800aed-c18c-4f9d-8be1-ab5294d140f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:02.912267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1362/2689] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:45:02 2022] DELETE /identity/v3/projects/3f7ea2694e7641088d957060a1001d86 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:02.918790 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0594c-fcb5-48fc-9859-dfb31f9a4d22 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:02.919198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0594c-fcb5-48fc-9859-dfb31f9a4d22 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:02.919648 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0594c-fcb5-48fc-9859-dfb31f9a4d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:02.944129 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-55f0594c-fcb5-48fc-9859-dfb31f9a4d22 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:02.946360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-55f0594c-fcb5-48fc-9859-dfb31f9a4d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:03.022862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1328/2690] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:03.030903 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aee484-06a6-4dad-8e5e-0bd2f2dc8465 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:03.044171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aee484-06a6-4dad-8e5e-0bd2f2dc8465 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:03.044171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aee484-06a6-4dad-8e5e-0bd2f2dc8465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:03.052954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f5aee484-06a6-4dad-8e5e-0bd2f2dc8465 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:03.054213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f5aee484-06a6-4dad-8e5e-0bd2f2dc8465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:03.063160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1363/2691] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:03.067798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0594c-fcb5-48fc-9859-dfb31f9a4d22 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:03.084123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0594c-fcb5-48fc-9859-dfb31f9a4d22 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:03.091406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec40bc-3293-4392-8ea2-f11ce5339554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:03.092125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec40bc-3293-4392-8ea2-f11ce5339554 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:03.092489 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec40bc-3293-4392-8ea2-f11ce5339554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:03.092830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec40bc-3293-4392-8ea2-f11ce5339554 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:03.093538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec40bc-3293-4392-8ea2-f11ce5339554 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:45:03.093909 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec40bc-3293-4392-8ea2-f11ce5339554 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:03.095564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec40bc-3293-4392-8ea2-f11ce5339554 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:03.116827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1329/2692] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:45:03.119033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aee484-06a6-4dad-8e5e-0bd2f2dc8465 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:03.135106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aee484-06a6-4dad-8e5e-0bd2f2dc8465 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:03.142369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db3016-5f43-43b4-8237-ca9a7bc2758a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:03.143259 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db3016-5f43-43b4-8237-ca9a7bc2758a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:03.143869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db3016-5f43-43b4-8237-ca9a7bc2758a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:03.143987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db3016-5f43-43b4-8237-ca9a7bc2758a admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:03.144961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db3016-5f43-43b4-8237-ca9a7bc2758a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:03.146892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db3016-5f43-43b4-8237-ca9a7bc2758a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:03.160792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:03.265226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1364/2693] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:45:03 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:45:03.266769 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec40bc-3293-4392-8ea2-f11ce5339554 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:03.278989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec40bc-3293-4392-8ea2-f11ce5339554 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:03.324423 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:03.347410 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d97740-dd8d-4341-b799-b4b5d9f623f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:03.348451 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97740-dd8d-4341-b799-b4b5d9f623f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:03.348801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97740-dd8d-4341-b799-b4b5d9f623f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:03.349185 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97740-dd8d-4341-b799-b4b5d9f623f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:03.349951 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d97740-dd8d-4341-b799-b4b5d9f623f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:03.352631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d97740-dd8d-4341-b799-b4b5d9f623f8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:03.472290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1330/2694] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:45:03 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:45:03.473413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db3016-5f43-43b4-8237-ca9a7bc2758a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:03.486726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db3016-5f43-43b4-8237-ca9a7bc2758a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:03.491898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3dd17b-f9f9-4bf8-bf7c-e8f431473a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:03.493715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dd17b-f9f9-4bf8-bf7c-e8f431473a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:03.495361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dd17b-f9f9-4bf8-bf7c-e8f431473a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:03.495361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dd17b-f9f9-4bf8-bf7c-e8f431473a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:03.495565 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3dd17b-f9f9-4bf8-bf7c-e8f431473a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:03.497369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3dd17b-f9f9-4bf8-bf7c-e8f431473a62 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:03.514319 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d97740-dd8d-4341-b799-b4b5d9f623f8 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:03.515450 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1365/2695] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:45:03.525500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d97740-dd8d-4341-b799-b4b5d9f623f8 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:03.530731 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91948941-7eef-4da5-8934-14c51c0ec63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:03.531412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91948941-7eef-4da5-8934-14c51c0ec63b admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:03.531756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91948941-7eef-4da5-8934-14c51c0ec63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:03.532139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91948941-7eef-4da5-8934-14c51c0ec63b admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:03.532854 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91948941-7eef-4da5-8934-14c51c0ec63b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:03.534626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91948941-7eef-4da5-8934-14c51c0ec63b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:03.550602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1331/2696] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:45:03.556027 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3dd17b-f9f9-4bf8-bf7c-e8f431473a62 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:03.569356 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3dd17b-f9f9-4bf8-bf7c-e8f431473a62 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:03.574344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b428d-ae8c-43d7-be1d-5221398e4f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:03.576162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b428d-ae8c-43d7-be1d-5221398e4f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:03.576162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b428d-ae8c-43d7-be1d-5221398e4f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:03.576162 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b428d-ae8c-43d7-be1d-5221398e4f06 admin admin] PATH_INFO: `/v3/projects/dd66028a147f42e1aa29abd7c22d00f9/users/990056333138468c99292c3f00605e6c/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:03.595150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b428d-ae8c-43d7-be1d-5221398e4f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd66028a147f42e1aa29abd7c22d00f9, user_id=990056333138468c99292c3f00605e6c, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:03.598295 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b428d-ae8c-43d7-be1d-5221398e4f06 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:03.676273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1366/2697] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:45:03 2022] PUT /identity/v3/projects/dd66028a147f42e1aa29abd7c22d00f9/users/990056333138468c99292c3f00605e6c/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:03.682216 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a07cf3-9f19-45e8-86cc-085278cac0ee None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:03.682556 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a07cf3-9f19-45e8-86cc-085278cac0ee None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:03.682972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a07cf3-9f19-45e8-86cc-085278cac0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:03.710055 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-66a07cf3-9f19-45e8-86cc-085278cac0ee None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:03.713021 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-66a07cf3-9f19-45e8-86cc-085278cac0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:03.805014 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1332/2698] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:03.813722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b428d-ae8c-43d7-be1d-5221398e4f06 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:03.816120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d64b428d-ae8c-43d7-be1d-5221398e4f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:03.890661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b428d-ae8c-43d7-be1d-5221398e4f06 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:03.894753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed99fd7-9c8b-45ec-92e5-de3f29b6367b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:03.895488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed99fd7-9c8b-45ec-92e5-de3f29b6367b service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:03.895839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed99fd7-9c8b-45ec-92e5-de3f29b6367b service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:03.896167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed99fd7-9c8b-45ec-92e5-de3f29b6367b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:03.908684 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed99fd7-9c8b-45ec-92e5-de3f29b6367b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:03.910548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed99fd7-9c8b-45ec-92e5-de3f29b6367b service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:03.922527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1367/2699] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:45:06.940870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e3c34-0e35-4758-a7da-040576dc6528 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:06.943667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e3c34-0e35-4758-a7da-040576dc6528 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:06.943667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e3c34-0e35-4758-a7da-040576dc6528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:06.963203 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-964e3c34-0e35-4758-a7da-040576dc6528 None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:06.966642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-964e3c34-0e35-4758-a7da-040576dc6528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:07.026838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1333/2700] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 22:45:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:07.039302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed99fd7-9c8b-45ec-92e5-de3f29b6367b service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:07.040914 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3ed99fd7-9c8b-45ec-92e5-de3f29b6367b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:07.125697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed99fd7-9c8b-45ec-92e5-de3f29b6367b service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:07.131232 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e07823-1c7f-49e6-bc34-aa7e4cc4a44d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:07.131947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e07823-1c7f-49e6-bc34-aa7e4cc4a44d service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:07.132280 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e07823-1c7f-49e6-bc34-aa7e4cc4a44d service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:07.133353 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e07823-1c7f-49e6-bc34-aa7e4cc4a44d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:07.143741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e07823-1c7f-49e6-bc34-aa7e4cc4a44d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:07.145291 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e07823-1c7f-49e6-bc34-aa7e4cc4a44d service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:07.160205 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1368/2701] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:45:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 07 22:45:09.677773 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cf0f3-70a5-42ee-9fbe-b89716e3ea41 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:09.678312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cf0f3-70a5-42ee-9fbe-b89716e3ea41 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:09.678481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cf0f3-70a5-42ee-9fbe-b89716e3ea41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:09.713833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d48cf0f3-70a5-42ee-9fbe-b89716e3ea41 None None] MFA Rules not processed for user `990056333138468c99292c3f00605e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:09.715051 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d48cf0f3-70a5-42ee-9fbe-b89716e3ea41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:09.791411 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:09.803600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1334/2702] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:09.813301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e07823-1c7f-49e6-bc34-aa7e4cc4a44d service swift] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:09.814985 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e9e07823-1c7f-49e6-bc34-aa7e4cc4a44d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:09.872970 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:09.899778 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e07823-1c7f-49e6-bc34-aa7e4cc4a44d service swift] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:09.906323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed557d4-798e-44f1-aa0d-e29f65c4aab4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:09.906951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed557d4-798e-44f1-aa0d-e29f65c4aab4 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:09.907255 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed557d4-798e-44f1-aa0d-e29f65c4aab4 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:09.907529 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed557d4-798e-44f1-aa0d-e29f65c4aab4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:09.917213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed557d4-798e-44f1-aa0d-e29f65c4aab4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:09.928448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed557d4-798e-44f1-aa0d-e29f65c4aab4 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:09.928448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1369/2703] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:45:10.415046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cf0f3-70a5-42ee-9fbe-b89716e3ea41 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:10.416562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d48cf0f3-70a5-42ee-9fbe-b89716e3ea41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:10.432659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cf0f3-70a5-42ee-9fbe-b89716e3ea41 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:10.438768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c51eaa-00ab-4fd1-8c95-aeb8ee41e1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:10.439408 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c51eaa-00ab-4fd1-8c95-aeb8ee41e1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:10.439686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c51eaa-00ab-4fd1-8c95-aeb8ee41e1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:10.439958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c51eaa-00ab-4fd1-8c95-aeb8ee41e1b7 admin admin] PATH_INFO: `/v3/users/269193a67b504842b3858022b378ae2a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:10.441145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c51eaa-00ab-4fd1-8c95-aeb8ee41e1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=269193a67b504842b3858022b378ae2a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:10.443193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c51eaa-00ab-4fd1-8c95-aeb8ee41e1b7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:10.634190 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b5c51eaa-00ab-4fd1-8c95-aeb8ee41e1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269193a67b504842b3858022b378ae2a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:10.707000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b5c51eaa-00ab-4fd1-8c95-aeb8ee41e1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269193a67b504842b3858022b378ae2a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:10.724402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b5c51eaa-00ab-4fd1-8c95-aeb8ee41e1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269193a67b504842b3858022b378ae2a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:10.725293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b5c51eaa-00ab-4fd1-8c95-aeb8ee41e1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269193a67b504842b3858022b378ae2a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:10.726742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b5c51eaa-00ab-4fd1-8c95-aeb8ee41e1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269193a67b504842b3858022b378ae2a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:10.744106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1335/2704] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:45:10 2022] DELETE /identity/v3/users/269193a67b504842b3858022b378ae2a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:10.991578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed557d4-798e-44f1-aa0d-e29f65c4aab4 service nova] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:10.995978 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed557d4-798e-44f1-aa0d-e29f65c4aab4 service nova] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:11.074160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:11.074469 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:11.074753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:11.075025 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] PATH_INFO: `/v3/projects/2648957a96504832a5693026f9d99c58` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:11.076480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2648957a96504832a5693026f9d99c58)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:11.078335 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:11.189527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2648957a96504832a5693026f9d99c58', 'request_id': 'req-34e99817-495e-4f53-a2ea-b71e6c76872c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:11.199685 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2648957a96504832a5693026f9d99c58', 'request_id': 'req-34e99817-495e-4f53-a2ea-b71e6c76872c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:11.253427 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2648957a96504832a5693026f9d99c58', 'request_id': 'req-34e99817-495e-4f53-a2ea-b71e6c76872c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:11.265711 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] The token cache is being invalidate because project 2648957a96504832a5693026f9d99c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:45:11.266007 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:11.266668 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-34e99817-495e-4f53-a2ea-b71e6c76872c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:11.268219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1370/2705] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:45:10 2022] DELETE /identity/v3/projects/2648957a96504832a5693026f9d99c58 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:11.274966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb28099-0d4a-493b-9d69-74cac97a8776 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:11.275293 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb28099-0d4a-493b-9d69-74cac97a8776 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:11.275596 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb28099-0d4a-493b-9d69-74cac97a8776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:11.297605 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3bb28099-0d4a-493b-9d69-74cac97a8776 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:11.298845 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3bb28099-0d4a-493b-9d69-74cac97a8776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:11.368113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1336/2706] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:45:11.375907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8c9b3-74c5-4f69-976c-3e4215b55a4a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:11.376230 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8c9b3-74c5-4f69-976c-3e4215b55a4a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:11.376511 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8c9b3-74c5-4f69-976c-3e4215b55a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:11.396740 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2cd8c9b3-74c5-4f69-976c-3e4215b55a4a None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:11.398518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2cd8c9b3-74c5-4f69-976c-3e4215b55a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:11.407265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1371/2707] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:11.411505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb28099-0d4a-493b-9d69-74cac97a8776 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:11.422502 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb28099-0d4a-493b-9d69-74cac97a8776 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:11.427338 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8ca0c-58ef-4208-8e0c-30e5b6b984d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:11.427954 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8ca0c-58ef-4208-8e0c-30e5b6b984d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:11.428266 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8ca0c-58ef-4208-8e0c-30e5b6b984d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:11.428545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8ca0c-58ef-4208-8e0c-30e5b6b984d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:11.429307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8ca0c-58ef-4208-8e0c-30e5b6b984d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:45:11.429734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8ca0c-58ef-4208-8e0c-30e5b6b984d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:11.431471 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8ca0c-58ef-4208-8e0c-30e5b6b984d9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:11.446725 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1337/2708] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:45:11.450218 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8c9b3-74c5-4f69-976c-3e4215b55a4a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:11.464341 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8c9b3-74c5-4f69-976c-3e4215b55a4a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:11.470649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813297f-13a5-4a04-9493-143fac8702cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:11.471370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813297f-13a5-4a04-9493-143fac8702cf admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:11.471716 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813297f-13a5-4a04-9493-143fac8702cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:11.472118 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813297f-13a5-4a04-9493-143fac8702cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:11.472838 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813297f-13a5-4a04-9493-143fac8702cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:11.474891 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813297f-13a5-4a04-9493-143fac8702cf admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:11.562967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1372/2709] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:45:11 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:45:11.567045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8ca0c-58ef-4208-8e0c-30e5b6b984d9 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:11.579846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8ca0c-58ef-4208-8e0c-30e5b6b984d9 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:11.635077 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:11.644507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f343f-d5f1-4f46-93ef-d3f03e3cb890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:11.645119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f343f-d5f1-4f46-93ef-d3f03e3cb890 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:11.645429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f343f-d5f1-4f46-93ef-d3f03e3cb890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:11.645706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f343f-d5f1-4f46-93ef-d3f03e3cb890 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:11.646364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f343f-d5f1-4f46-93ef-d3f03e3cb890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:11.649043 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f343f-d5f1-4f46-93ef-d3f03e3cb890 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:11.740670 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1338/2710] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:45:11 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:45:11.743267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813297f-13a5-4a04-9493-143fac8702cf admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:11.756272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813297f-13a5-4a04-9493-143fac8702cf admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:11.762581 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d96914-8aaf-46ff-8d85-57fd7cac5f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:11.763904 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d96914-8aaf-46ff-8d85-57fd7cac5f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:11.764368 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d96914-8aaf-46ff-8d85-57fd7cac5f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:11.765225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d96914-8aaf-46ff-8d85-57fd7cac5f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:11.765365 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d96914-8aaf-46ff-8d85-57fd7cac5f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:11.766921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d96914-8aaf-46ff-8d85-57fd7cac5f40 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:11.778613 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1373/2711] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:45:11.782723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f343f-d5f1-4f46-93ef-d3f03e3cb890 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:11.793124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f343f-d5f1-4f46-93ef-d3f03e3cb890 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:11.798008 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473be15c-1982-4ad2-8da5-17e7fbf39171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:11.798863 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473be15c-1982-4ad2-8da5-17e7fbf39171 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:11.799149 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473be15c-1982-4ad2-8da5-17e7fbf39171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:11.799456 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473be15c-1982-4ad2-8da5-17e7fbf39171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:11.800170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473be15c-1982-4ad2-8da5-17e7fbf39171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:11.801783 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473be15c-1982-4ad2-8da5-17e7fbf39171 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:11.816065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1339/2712] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:45:11.821396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d96914-8aaf-46ff-8d85-57fd7cac5f40 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:11.831799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d96914-8aaf-46ff-8d85-57fd7cac5f40 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:11.837443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd10495-4443-44af-963b-c8ea02e283bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:11.838193 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10495-4443-44af-963b-c8ea02e283bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:11.838439 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10495-4443-44af-963b-c8ea02e283bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:11.838649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10495-4443-44af-963b-c8ea02e283bf admin admin] PATH_INFO: `/v3/projects/b7af672d47194fa19472fe8322424616/users/47d33d20165a4526a97263f43aa9a5e4/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:11.856385 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd10495-4443-44af-963b-c8ea02e283bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7af672d47194fa19472fe8322424616, user_id=47d33d20165a4526a97263f43aa9a5e4, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:11.859178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd10495-4443-44af-963b-c8ea02e283bf admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:11.941307 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:11.946782 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1374/2713] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:45:11 2022] PUT /identity/v3/projects/b7af672d47194fa19472fe8322424616/users/47d33d20165a4526a97263f43aa9a5e4/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:45:12.015434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6224b8-561c-495b-aa2f-f03ebd41d5f2 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:12.015724 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6224b8-561c-495b-aa2f-f03ebd41d5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:12.016478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6224b8-561c-495b-aa2f-f03ebd41d5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:12.045668 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-db6224b8-561c-495b-aa2f-f03ebd41d5f2 None None] MFA Rules not processed for user `47d33d20165a4526a97263f43aa9a5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:12.046895 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-db6224b8-561c-495b-aa2f-f03ebd41d5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:12.118837 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1340/2714] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:12.131661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd10495-4443-44af-963b-c8ea02e283bf admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:12.133845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-acd10495-4443-44af-963b-c8ea02e283bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:12.198578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd10495-4443-44af-963b-c8ea02e283bf admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:12.202339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32389f2f-d6bf-4370-b73b-b218938cb0b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:12.202995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32389f2f-d6bf-4370-b73b-b218938cb0b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:12.203274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32389f2f-d6bf-4370-b73b-b218938cb0b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:12.203549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32389f2f-d6bf-4370-b73b-b218938cb0b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:12.216746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32389f2f-d6bf-4370-b73b-b218938cb0b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:12.218852 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32389f2f-d6bf-4370-b73b-b218938cb0b2 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:12.233244 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1375/2715] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:45:12.568262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc1a07-d697-4b40-a02e-f2ae8064137f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:12.568545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc1a07-d697-4b40-a02e-f2ae8064137f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:12.568823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc1a07-d697-4b40-a02e-f2ae8064137f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:12.593097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-60dc1a07-d697-4b40-a02e-f2ae8064137f None None] MFA Rules not processed for user `990056333138468c99292c3f00605e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:12.593842 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-60dc1a07-d697-4b40-a02e-f2ae8064137f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:12.655156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1341/2716] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:15.320867 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32389f2f-d6bf-4370-b73b-b218938cb0b2 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:15.384217 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32389f2f-d6bf-4370-b73b-b218938cb0b2 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:15.389012 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:15.389631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:15.389911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:15.390186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] PATH_INFO: `/v3/users/47d33d20165a4526a97263f43aa9a5e4` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:15.391324 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d33d20165a4526a97263f43aa9a5e4)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:15.393267 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:15.552803 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d33d20165a4526a97263f43aa9a5e4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:15.636435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d33d20165a4526a97263f43aa9a5e4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:15.649083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d33d20165a4526a97263f43aa9a5e4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:15.656285 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d33d20165a4526a97263f43aa9a5e4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:15.656975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d33d20165a4526a97263f43aa9a5e4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:15.662972 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1376/2717] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:45:15 2022] DELETE /identity/v3/users/47d33d20165a4526a97263f43aa9a5e4 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:15.667569 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fc086-5ebd-42fa-aa75-8f9293bfa5ab None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:15.669700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fc086-5ebd-42fa-aa75-8f9293bfa5ab None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:15.669700 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fc086-5ebd-42fa-aa75-8f9293bfa5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:15.687977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d57fc086-5ebd-42fa-aa75-8f9293bfa5ab None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:15.689215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d57fc086-5ebd-42fa-aa75-8f9293bfa5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:15.761244 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1342/2718] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:15.767807 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:15.769310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:15.797448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:15.829756 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d5546-6ce6-4019-9154-ef8c99d02d0e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:15.833454 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cba166-7c8a-4350-9c07-db14fc5b4787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:15.834207 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba166-7c8a-4350-9c07-db14fc5b4787 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:15.834647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba166-7c8a-4350-9c07-db14fc5b4787 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:15.834999 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba166-7c8a-4350-9c07-db14fc5b4787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:15.845324 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cba166-7c8a-4350-9c07-db14fc5b4787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:15.847275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cba166-7c8a-4350-9c07-db14fc5b4787 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:15.859591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1377/2719] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:45:16.302100 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fc086-5ebd-42fa-aa75-8f9293bfa5ab None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:16.303589 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d57fc086-5ebd-42fa-aa75-8f9293bfa5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:16.323520 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fc086-5ebd-42fa-aa75-8f9293bfa5ab None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:16.329738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:16.330398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:16.330671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:16.330941 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] PATH_INFO: `/v3/projects/b7af672d47194fa19472fe8322424616` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:16.332434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7af672d47194fa19472fe8322424616)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:16.334588 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:16.336420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:16.458210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7af672d47194fa19472fe8322424616', 'request_id': 'req-d0d63531-5a7c-42e3-a496-81a1c9013ced'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:16.465157 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7af672d47194fa19472fe8322424616', 'request_id': 'req-d0d63531-5a7c-42e3-a496-81a1c9013ced'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:16.527717 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7af672d47194fa19472fe8322424616', 'request_id': 'req-d0d63531-5a7c-42e3-a496-81a1c9013ced'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:16.540505 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] The token cache is being invalidate because project b7af672d47194fa19472fe8322424616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:45:16.540795 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:16.541538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d0d63531-5a7c-42e3-a496-81a1c9013ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:16.543364 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1343/2720] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:45:16 2022] DELETE /identity/v3/projects/b7af672d47194fa19472fe8322424616 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:16.548063 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4786a-8ca7-49ae-ae23-37a2683cebfa None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:16.548533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4786a-8ca7-49ae-ae23-37a2683cebfa None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:16.548736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4786a-8ca7-49ae-ae23-37a2683cebfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:16.569104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0cc4786a-8ca7-49ae-ae23-37a2683cebfa None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:16.571242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0cc4786a-8ca7-49ae-ae23-37a2683cebfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:16.637948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1378/2721] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:16.645582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e68147-d0c6-429d-8ace-5998dde32ad4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:16.645898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e68147-d0c6-429d-8ace-5998dde32ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:16.646178 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e68147-d0c6-429d-8ace-5998dde32ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:16.669046 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-32e68147-d0c6-429d-8ace-5998dde32ad4 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:16.671020 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-32e68147-d0c6-429d-8ace-5998dde32ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:16.679919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1344/2722] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:16.684313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4786a-8ca7-49ae-ae23-37a2683cebfa None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:16.694331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4786a-8ca7-49ae-ae23-37a2683cebfa None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:16.699622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052a60-8f43-47dc-9779-5d9d5b341200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:16.700289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052a60-8f43-47dc-9779-5d9d5b341200 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:16.700570 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052a60-8f43-47dc-9779-5d9d5b341200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:16.700934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052a60-8f43-47dc-9779-5d9d5b341200 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:16.701599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84052a60-8f43-47dc-9779-5d9d5b341200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:45:16.701907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84052a60-8f43-47dc-9779-5d9d5b341200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:16.703539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84052a60-8f43-47dc-9779-5d9d5b341200 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:16.716799 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1379/2723] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:45:16.721052 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e68147-d0c6-429d-8ace-5998dde32ad4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:16.731798 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e68147-d0c6-429d-8ace-5998dde32ad4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:16.736855 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353bc7f-96c9-4345-896f-e1477988f9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:16.737561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353bc7f-96c9-4345-896f-e1477988f9fc admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:16.737912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353bc7f-96c9-4345-896f-e1477988f9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:16.738313 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353bc7f-96c9-4345-896f-e1477988f9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:16.739049 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353bc7f-96c9-4345-896f-e1477988f9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:16.740996 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353bc7f-96c9-4345-896f-e1477988f9fc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:16.814025 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1345/2724] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:45:16 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:45:16.818839 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052a60-8f43-47dc-9779-5d9d5b341200 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:16.831840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052a60-8f43-47dc-9779-5d9d5b341200 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:16.902890 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168e87b-ffaa-4099-81c6-6aeb87016fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:16.904132 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168e87b-ffaa-4099-81c6-6aeb87016fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:16.904792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168e87b-ffaa-4099-81c6-6aeb87016fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:16.904792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168e87b-ffaa-4099-81c6-6aeb87016fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:16.905617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168e87b-ffaa-4099-81c6-6aeb87016fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:16.907586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168e87b-ffaa-4099-81c6-6aeb87016fd6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:16.973281 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1380/2725] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:45:16 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:45:16.974817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353bc7f-96c9-4345-896f-e1477988f9fc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:16.986389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353bc7f-96c9-4345-896f-e1477988f9fc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:16.991316 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2eda8b-fadf-483e-bf59-46efaadd81db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:16.992172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2eda8b-fadf-483e-bf59-46efaadd81db admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:16.992644 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2eda8b-fadf-483e-bf59-46efaadd81db admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:16.992928 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2eda8b-fadf-483e-bf59-46efaadd81db admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:16.993649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2eda8b-fadf-483e-bf59-46efaadd81db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:16.995260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2eda8b-fadf-483e-bf59-46efaadd81db admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:17.005667 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1346/2726] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:45:17.010441 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168e87b-ffaa-4099-81c6-6aeb87016fd6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:17.021014 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168e87b-ffaa-4099-81c6-6aeb87016fd6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:17.027069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37e468-5bb7-4bfd-b0dd-b90797dd88de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:17.027898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37e468-5bb7-4bfd-b0dd-b90797dd88de admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:17.028221 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37e468-5bb7-4bfd-b0dd-b90797dd88de admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:17.028496 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37e468-5bb7-4bfd-b0dd-b90797dd88de admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:17.029155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37e468-5bb7-4bfd-b0dd-b90797dd88de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:17.030859 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37e468-5bb7-4bfd-b0dd-b90797dd88de admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:17.042775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1381/2727] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:45:17.047108 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2eda8b-fadf-483e-bf59-46efaadd81db admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:17.058113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2eda8b-fadf-483e-bf59-46efaadd81db admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:17.062890 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbccdcc-4204-44ab-8283-d7e8c513bd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:17.063545 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbccdcc-4204-44ab-8283-d7e8c513bd53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:17.063822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbccdcc-4204-44ab-8283-d7e8c513bd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:17.064195 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbccdcc-4204-44ab-8283-d7e8c513bd53 admin admin] PATH_INFO: `/v3/projects/ad0a4d9a434e41cab6cdc7e20fd8b0bb/users/097f69347f5a460ea7650d9854260e97/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:17.081275 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbccdcc-4204-44ab-8283-d7e8c513bd53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0a4d9a434e41cab6cdc7e20fd8b0bb, user_id=097f69347f5a460ea7650d9854260e97, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:17.084056 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbccdcc-4204-44ab-8283-d7e8c513bd53 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:17.162858 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1347/2728] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:45:17 2022] PUT /identity/v3/projects/ad0a4d9a434e41cab6cdc7e20fd8b0bb/users/097f69347f5a460ea7650d9854260e97/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:17.173566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f84f4-dd75-44b6-b5ec-e80240aff406 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:17.173927 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f84f4-dd75-44b6-b5ec-e80240aff406 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:17.174299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f84f4-dd75-44b6-b5ec-e80240aff406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:17.203448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-033f84f4-dd75-44b6-b5ec-e80240aff406 None None] MFA Rules not processed for user `097f69347f5a460ea7650d9854260e97`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:17.204746 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-033f84f4-dd75-44b6-b5ec-e80240aff406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:17.223134 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:17.272584 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1382/2729] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:17.283776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbccdcc-4204-44ab-8283-d7e8c513bd53 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:17.285938 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ecbccdcc-4204-44ab-8283-d7e8c513bd53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:17.346856 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbccdcc-4204-44ab-8283-d7e8c513bd53 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:17.351083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398adf25-31db-4966-9a2f-5337e6530b03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:17.351795 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398adf25-31db-4966-9a2f-5337e6530b03 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:17.352106 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398adf25-31db-4966-9a2f-5337e6530b03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:17.352376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398adf25-31db-4966-9a2f-5337e6530b03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:17.361119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398adf25-31db-4966-9a2f-5337e6530b03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:17.362601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398adf25-31db-4966-9a2f-5337e6530b03 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:17.372745 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1348/2730] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:45:19.115408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0dda3-062c-478e-8d41-fea06e040540 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:19.115937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0dda3-062c-478e-8d41-fea06e040540 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:19.116116 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0dda3-062c-478e-8d41-fea06e040540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:19.135538 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-07b0dda3-062c-478e-8d41-fea06e040540 None None] MFA Rules not processed for user `097f69347f5a460ea7650d9854260e97`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:19.137022 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-07b0dda3-062c-478e-8d41-fea06e040540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:19.143834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1383/2731] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:27.368949 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:29.130754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=21 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:35.339720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398adf25-31db-4966-9a2f-5337e6530b03 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:35.341297 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-398adf25-31db-4966-9a2f-5337e6530b03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:35.513982 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398adf25-31db-4966-9a2f-5337e6530b03 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:35.519905 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328ec5c-5964-4608-9d55-9a912c5f9bea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:35.522187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328ec5c-5964-4608-9d55-9a912c5f9bea service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:35.522187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328ec5c-5964-4608-9d55-9a912c5f9bea service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:35.522187 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328ec5c-5964-4608-9d55-9a912c5f9bea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:35.523208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1328ec5c-5964-4608-9d55-9a912c5f9bea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:35.535905 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328ec5c-5964-4608-9d55-9a912c5f9bea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:35.537497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328ec5c-5964-4608-9d55-9a912c5f9bea service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:35.547967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1349/2732] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 22:45:37.428954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05266c-494d-4fa2-81ba-d212c9b59f9f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:37.429518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05266c-494d-4fa2-81ba-d212c9b59f9f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:37.429518 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05266c-494d-4fa2-81ba-d212c9b59f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:37.457763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4a05266c-494d-4fa2-81ba-d212c9b59f9f None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:37.458962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4a05266c-494d-4fa2-81ba-d212c9b59f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:37.528623 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1384/2733] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:45:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:37.541111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328ec5c-5964-4608-9d55-9a912c5f9bea service placement] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:37.542506 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1328ec5c-5964-4608-9d55-9a912c5f9bea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:37.615663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328ec5c-5964-4608-9d55-9a912c5f9bea service placement] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:37.621158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db440a8a-56d2-4969-a5d9-edc618d297d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:37.621741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db440a8a-56d2-4969-a5d9-edc618d297d3 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:37.622015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db440a8a-56d2-4969-a5d9-edc618d297d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:37.622285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db440a8a-56d2-4969-a5d9-edc618d297d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:37.630351 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db440a8a-56d2-4969-a5d9-edc618d297d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:37.634570 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db440a8a-56d2-4969-a5d9-edc618d297d3 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:37.640860 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1350/2734] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:45:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:45:37.715566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b881061-c69a-4be7-954a-3638bad018a4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:37.715898 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b881061-c69a-4be7-954a-3638bad018a4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:37.716190 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b881061-c69a-4be7-954a-3638bad018a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:37.735993 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4b881061-c69a-4be7-954a-3638bad018a4 None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:37.737331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4b881061-c69a-4be7-954a-3638bad018a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:37.745648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1385/2735] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 22:45:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:37.763320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db440a8a-56d2-4969-a5d9-edc618d297d3 service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:37.775558 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db440a8a-56d2-4969-a5d9-edc618d297d3 service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:37.780752 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637992a-e2fa-42b8-a738-692e53a85ae2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:37.781345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637992a-e2fa-42b8-a738-692e53a85ae2 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:37.783814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637992a-e2fa-42b8-a738-692e53a85ae2 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:37.783814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637992a-e2fa-42b8-a738-692e53a85ae2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:37.790654 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637992a-e2fa-42b8-a738-692e53a85ae2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:37.792296 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637992a-e2fa-42b8-a738-692e53a85ae2 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:37.795850 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:37.800978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1351/2736] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:45:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 07 22:45:38.638797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f60088-a34b-458a-aded-c03409dcf4f6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:38.639311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f60088-a34b-458a-aded-c03409dcf4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:38.639417 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f60088-a34b-458a-aded-c03409dcf4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:38.659369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-43f60088-a34b-458a-aded-c03409dcf4f6 None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:38.660596 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-43f60088-a34b-458a-aded-c03409dcf4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:38.668238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1386/2737] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 22:45:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:45:38.682958 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637992a-e2fa-42b8-a738-692e53a85ae2 service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:38.691709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637992a-e2fa-42b8-a738-692e53a85ae2 service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:38.696747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a547426-ce3a-4b80-8986-993460aae886 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:38.697325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a547426-ce3a-4b80-8986-993460aae886 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:38.697602 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a547426-ce3a-4b80-8986-993460aae886 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:38.697869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a547426-ce3a-4b80-8986-993460aae886 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:38.705894 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a547426-ce3a-4b80-8986-993460aae886 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:38.709657 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a547426-ce3a-4b80-8986-993460aae886 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:38.715647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1352/2738] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:45:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:45:39.433150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d84ef9-785f-4c65-9467-41b9eb6cfe9e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:39.433437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d84ef9-785f-4c65-9467-41b9eb6cfe9e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:39.433717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d84ef9-785f-4c65-9467-41b9eb6cfe9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:39.454184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c7d84ef9-785f-4c65-9467-41b9eb6cfe9e None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:39.455419 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c7d84ef9-785f-4c65-9467-41b9eb6cfe9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:39.463560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1387/2739] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 22:45:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:45:39.475952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a547426-ce3a-4b80-8986-993460aae886 service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:39.488074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a547426-ce3a-4b80-8986-993460aae886 service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:39.494558 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dbc84-2859-4b8e-9bde-719bb517ea36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:39.495472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dbc84-2859-4b8e-9bde-719bb517ea36 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:39.495871 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dbc84-2859-4b8e-9bde-719bb517ea36 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:39.496344 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dbc84-2859-4b8e-9bde-719bb517ea36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:39.505398 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3dbc84-2859-4b8e-9bde-719bb517ea36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:39.506965 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3dbc84-2859-4b8e-9bde-719bb517ea36 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:39.515991 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1353/2740] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:45:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:45:40.151002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cf49f-57e0-4671-9ec1-c342d7c68718 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:40.151428 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cf49f-57e0-4671-9ec1-c342d7c68718 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:40.151537 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cf49f-57e0-4671-9ec1-c342d7c68718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:40.173384 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d22cf49f-57e0-4671-9ec1-c342d7c68718 None None] MFA Rules not processed for user `77cb38c8c0dc4de3bbdeb611ffd53ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:40.175288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d22cf49f-57e0-4671-9ec1-c342d7c68718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:40.183916 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1388/2741] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 22:45:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:45:40.199977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dbc84-2859-4b8e-9bde-719bb517ea36 service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:40.213416 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dbc84-2859-4b8e-9bde-719bb517ea36 service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:40.220659 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444fdd4-ca0d-4fe9-a407-7b27bb7b0470 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86991778c2b47c58f10d9d9e3ca5e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:40.221900 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444fdd4-ca0d-4fe9-a407-7b27bb7b0470 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:40.222441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444fdd4-ca0d-4fe9-a407-7b27bb7b0470 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:40.222963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444fdd4-ca0d-4fe9-a407-7b27bb7b0470 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:40.235633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444fdd4-ca0d-4fe9-a407-7b27bb7b0470 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:40.237914 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444fdd4-ca0d-4fe9-a407-7b27bb7b0470 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:40.249833 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1354/2742] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 22:45:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 22:45:43.056242 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22cf49f-57e0-4671-9ec1-c342d7c68718 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:43.141158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22cf49f-57e0-4671-9ec1-c342d7c68718 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:43.145814 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:43.146441 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:43.146726 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:43.146999 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] PATH_INFO: `/v3/users/097f69347f5a460ea7650d9854260e97` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:43.148847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=097f69347f5a460ea7650d9854260e97)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:43.151150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:43.311431 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '097f69347f5a460ea7650d9854260e97'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:43.313331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:43.385981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '097f69347f5a460ea7650d9854260e97'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:43.399956 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '097f69347f5a460ea7650d9854260e97'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:43.407226 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '097f69347f5a460ea7650d9854260e97'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:43.407914 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '097f69347f5a460ea7650d9854260e97'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:43.414274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1389/2743] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:45:43 2022] DELETE /identity/v3/users/097f69347f5a460ea7650d9854260e97 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:43.418573 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4db37d-b0bb-4ebd-83ec-f250d6088525 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:43.419476 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4db37d-b0bb-4ebd-83ec-f250d6088525 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:43.419782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4db37d-b0bb-4ebd-83ec-f250d6088525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:43.461836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-aa4db37d-b0bb-4ebd-83ec-f250d6088525 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:45:43.463117 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-aa4db37d-b0bb-4ebd-83ec-f250d6088525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:43.524287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1355/2744] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:45:43.534168 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:43.535661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:43.615754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b2d90-4641-4a73-ac46-22d258b2fc5c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:43.620052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb018f-aef3-4c62-8a56-8cbf08648dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:43.623802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb018f-aef3-4c62-8a56-8cbf08648dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:43.623802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb018f-aef3-4c62-8a56-8cbf08648dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:43.623802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb018f-aef3-4c62-8a56-8cbf08648dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:43.630802 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb018f-aef3-4c62-8a56-8cbf08648dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:43.632265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb018f-aef3-4c62-8a56-8cbf08648dd6 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:43.643017 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1390/2745] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:45:43.933714 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4db37d-b0bb-4ebd-83ec-f250d6088525 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:43.935544 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-aa4db37d-b0bb-4ebd-83ec-f250d6088525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:43.954741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4db37d-b0bb-4ebd-83ec-f250d6088525 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:43.961824 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:43.963156 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:43.963644 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:43.963920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] PATH_INFO: `/v3/projects/ad0a4d9a434e41cab6cdc7e20fd8b0bb` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:43.965750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0a4d9a434e41cab6cdc7e20fd8b0bb)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:43.967665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:44.078524 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0a4d9a434e41cab6cdc7e20fd8b0bb', 'request_id': 'req-831cc36e-9e40-482d-85e5-2b35f422bd46'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:44.085002 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0a4d9a434e41cab6cdc7e20fd8b0bb', 'request_id': 'req-831cc36e-9e40-482d-85e5-2b35f422bd46'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:44.156600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0a4d9a434e41cab6cdc7e20fd8b0bb', 'request_id': 'req-831cc36e-9e40-482d-85e5-2b35f422bd46'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:44.169441 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] The token cache is being invalidate because project ad0a4d9a434e41cab6cdc7e20fd8b0bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:45:44.169746 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:44.170379 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:44.172206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1356/2746] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:45:43 2022] DELETE /identity/v3/projects/ad0a4d9a434e41cab6cdc7e20fd8b0bb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:46.018010 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb018f-aef3-4c62-8a56-8cbf08648dd6 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:46.019610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-15bb018f-aef3-4c62-8a56-8cbf08648dd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:46.092787 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb018f-aef3-4c62-8a56-8cbf08648dd6 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:46.144915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:46.144915 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:46.146684 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:46.164954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9fd5d-a95d-480a-a35e-0128d83c433d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:46.165572 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9fd5d-a95d-480a-a35e-0128d83c433d service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:46.165855 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9fd5d-a95d-480a-a35e-0128d83c433d service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:46.166130 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9fd5d-a95d-480a-a35e-0128d83c433d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:46.168592 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:46.168738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-17d9fd5d-a95d-480a-a35e-0128d83c433d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:46.180744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9fd5d-a95d-480a-a35e-0128d83c433d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:46.182351 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9fd5d-a95d-480a-a35e-0128d83c433d service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:46.192513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1391/2747] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:45:46.195670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9fd5d-a95d-480a-a35e-0128d83c433d service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:46.197236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-17d9fd5d-a95d-480a-a35e-0128d83c433d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:46.208260 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cc36e-9e40-482d-85e5-2b35f422bd46 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:46.214831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:46.215484 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:46.215827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:46.216154 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] PATH_INFO: `/v3/users/acf297d1f1f244dd997992583189c127` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:46.218122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf297d1f1f244dd997992583189c127)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:46.221005 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:46.256597 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9fd5d-a95d-480a-a35e-0128d83c433d service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:46.260835 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692934ff-352f-4d32-bd37-d6320dc1efa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:46.261443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692934ff-352f-4d32-bd37-d6320dc1efa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:46.261715 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692934ff-352f-4d32-bd37-d6320dc1efa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:46.262250 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692934ff-352f-4d32-bd37-d6320dc1efa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:46.263981 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-692934ff-352f-4d32-bd37-d6320dc1efa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:46.322320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692934ff-352f-4d32-bd37-d6320dc1efa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:46.323758 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692934ff-352f-4d32-bd37-d6320dc1efa4 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:46.332575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1392/2748] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:45:46.408901 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf297d1f1f244dd997992583189c127'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:46.477629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf297d1f1f244dd997992583189c127'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:46.490369 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf297d1f1f244dd997992583189c127'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:46.491112 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf297d1f1f244dd997992583189c127'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:46.492664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf297d1f1f244dd997992583189c127'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:46.505256 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1357/2749] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:45:46 2022] DELETE /identity/v3/users/acf297d1f1f244dd997992583189c127 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:45:46.753315 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692934ff-352f-4d32-bd37-d6320dc1efa4 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:46.765771 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692934ff-352f-4d32-bd37-d6320dc1efa4 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:46.826546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:46.827182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:46.827462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:46.827737 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] PATH_INFO: `/v3/projects/bcc9a378e7ba4206baaa7bcc683606ca` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:46.828905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcc9a378e7ba4206baaa7bcc683606ca)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:46.830817 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:46.932424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:46.948122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcc9a378e7ba4206baaa7bcc683606ca', 'request_id': 'req-b1bc8286-46c5-4c4d-8316-7a296b02450e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:46.954630 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcc9a378e7ba4206baaa7bcc683606ca', 'request_id': 'req-b1bc8286-46c5-4c4d-8316-7a296b02450e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:47.054980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcc9a378e7ba4206baaa7bcc683606ca', 'request_id': 'req-b1bc8286-46c5-4c4d-8316-7a296b02450e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:47.066602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] The token cache is being invalidate because project bcc9a378e7ba4206baaa7bcc683606ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:45:47.066908 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:47.067576 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:47.069135 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1393/2750] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:45:46 2022] DELETE /identity/v3/projects/bcc9a378e7ba4206baaa7bcc683606ca => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:47.073568 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:47.075062 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:47.089911 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac0b4c-08b9-4d52-a13f-b20ad5544367 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:47.151865 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:47.152591 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:47.152806 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:47.153080 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] PATH_INFO: `/v3/users/5eb7cd1bc8de49a8a7fc0db68e3eab58` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:47.154275 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eb7cd1bc8de49a8a7fc0db68e3eab58)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:47.156242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:47.342264 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:47.344155 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eb7cd1bc8de49a8a7fc0db68e3eab58'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:47.425827 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eb7cd1bc8de49a8a7fc0db68e3eab58'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:47.439806 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eb7cd1bc8de49a8a7fc0db68e3eab58'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:47.441361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eb7cd1bc8de49a8a7fc0db68e3eab58'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:47.442051 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eb7cd1bc8de49a8a7fc0db68e3eab58'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:47.455529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1358/2751] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:45:47 2022] DELETE /identity/v3/users/5eb7cd1bc8de49a8a7fc0db68e3eab58 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:47.903562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:47.905654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:47.920002 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc8286-46c5-4c4d-8316-7a296b02450e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:48.000122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:48.001062 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:48.001498 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:48.001945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] PATH_INFO: `/v3/projects/6aaf55537b3846c0a2cf60a710c38003` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:48.003275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aaf55537b3846c0a2cf60a710c38003)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:48.005601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:48.127404 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aaf55537b3846c0a2cf60a710c38003', 'request_id': 'req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:48.134949 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aaf55537b3846c0a2cf60a710c38003', 'request_id': 'req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:48.201619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aaf55537b3846c0a2cf60a710c38003', 'request_id': 'req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:48.213562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] The token cache is being invalidate because project 6aaf55537b3846c0a2cf60a710c38003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:45:48.213955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:48.214638 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:48.216174 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1394/2752] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:45:47 2022] DELETE /identity/v3/projects/6aaf55537b3846c0a2cf60a710c38003 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:45:51.778496 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:51.779980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:51.836158 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71bfef-a4b6-4d33-91b8-65541e1b9a6c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:51.882965 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16f7ca-3905-4308-ba94-3c0fee67117e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:51.883585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16f7ca-3905-4308-ba94-3c0fee67117e service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:51.883862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16f7ca-3905-4308-ba94-3c0fee67117e service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:51.884161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16f7ca-3905-4308-ba94-3c0fee67117e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:51.885872 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6e16f7ca-3905-4308-ba94-3c0fee67117e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:51.949867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16f7ca-3905-4308-ba94-3c0fee67117e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:51.951303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16f7ca-3905-4308-ba94-3c0fee67117e service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:51.960898 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1359/2753] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:45:55.388112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:55.392304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:55.496788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:55.518429 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc71fc-317e-44a6-9a02-4bbd5e076e13 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:55.525033 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098dfc99-84ba-45a6-bdcd-04d11d29d598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:55.525645 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dfc99-84ba-45a6-bdcd-04d11d29d598 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:55.527946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dfc99-84ba-45a6-bdcd-04d11d29d598 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:55.527946 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dfc99-84ba-45a6-bdcd-04d11d29d598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:55.528282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-098dfc99-84ba-45a6-bdcd-04d11d29d598 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:55.591901 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098dfc99-84ba-45a6-bdcd-04d11d29d598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:55.593435 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098dfc99-84ba-45a6-bdcd-04d11d29d598 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:55.603736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1395/2754] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:45:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:45:59.385215 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16f7ca-3905-4308-ba94-3c0fee67117e service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:45:59.386771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6e16f7ca-3905-4308-ba94-3c0fee67117e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:45:59.403075 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16f7ca-3905-4308-ba94-3c0fee67117e service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:45:59.405548 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:45:59.406205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:45:59.406481 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:45:59.406753 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] PATH_INFO: `/v3/users/d7f2dbb97068490d924ae132224dc8ff` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:45:59.407919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7f2dbb97068490d924ae132224dc8ff)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:45:59.409880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:45:59.432792 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:45:59.652214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7f2dbb97068490d924ae132224dc8ff'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:59.732533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7f2dbb97068490d924ae132224dc8ff'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:59.745913 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7f2dbb97068490d924ae132224dc8ff'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:59.746554 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7f2dbb97068490d924ae132224dc8ff'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:59.747163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7f2dbb97068490d924ae132224dc8ff'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:45:59.760477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1360/2755] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:45:59 2022] DELETE /identity/v3/users/d7f2dbb97068490d924ae132224dc8ff => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:46:00.027749 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098dfc99-84ba-45a6-bdcd-04d11d29d598 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:46:00.039142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098dfc99-84ba-45a6-bdcd-04d11d29d598 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:46:00.096203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:46:00.096826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:46:00.097106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:46:00.097379 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] PATH_INFO: `/v3/projects/097fb8408bfb479d87e7516a93970e61` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:46:00.098566 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=097fb8408bfb479d87e7516a93970e61)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:46:00.100563 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:46:00.220858 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097fb8408bfb479d87e7516a93970e61', 'request_id': 'req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:00.228894 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097fb8408bfb479d87e7516a93970e61', 'request_id': 'req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:00.317137 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097fb8408bfb479d87e7516a93970e61', 'request_id': 'req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:00.329069 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] The token cache is being invalidate because project 097fb8408bfb479d87e7516a93970e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:46:00.329360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:00.330085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:00.331595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1396/2756] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:46:00 2022] DELETE /identity/v3/projects/097fb8408bfb479d87e7516a93970e61 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:46:00.335671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:46:00.337128 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:46:00.349983 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba7ece-1515-423b-82de-4bf79677a644 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:46:00.408478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9d9ac-9475-4c5b-897b-638ae01c5518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:46:00.409166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d9ac-9475-4c5b-897b-638ae01c5518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:46:00.409421 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d9ac-9475-4c5b-897b-638ae01c5518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:46:00.409695 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d9ac-9475-4c5b-897b-638ae01c5518 admin admin] PATH_INFO: `/v3/users/a7567b06d54442cca9b337d76aaafd5c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:46:00.410831 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9d9ac-9475-4c5b-897b-638ae01c5518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7567b06d54442cca9b337d76aaafd5c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:46:00.412789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9d9ac-9475-4c5b-897b-638ae01c5518 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:46:00.651324 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9cb9d9ac-9475-4c5b-897b-638ae01c5518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7567b06d54442cca9b337d76aaafd5c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:00.722749 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9cb9d9ac-9475-4c5b-897b-638ae01c5518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7567b06d54442cca9b337d76aaafd5c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:00.736323 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9cb9d9ac-9475-4c5b-897b-638ae01c5518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7567b06d54442cca9b337d76aaafd5c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:00.736995 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9cb9d9ac-9475-4c5b-897b-638ae01c5518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7567b06d54442cca9b337d76aaafd5c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:00.737626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9cb9d9ac-9475-4c5b-897b-638ae01c5518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7567b06d54442cca9b337d76aaafd5c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:00.750341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1361/2757] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:46:00 2022] DELETE /identity/v3/users/a7567b06d54442cca9b337d76aaafd5c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:46:00.982241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:46:00.996227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:46:00.997258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c4657-dfb0-498e-ad5a-c1591a1e3bac admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:46:01.017340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:46:01.059822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:46:01.060554 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:46:01.060832 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:46:01.061125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] PATH_INFO: `/v3/projects/643f0ddc528d45f1ae5d7a7396b36684` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:46:01.062270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=643f0ddc528d45f1ae5d7a7396b36684)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:46:01.064218 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:46:01.204147 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643f0ddc528d45f1ae5d7a7396b36684', 'request_id': 'req-6d991fd5-5008-431c-b7c7-2203379a48f3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:01.210501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643f0ddc528d45f1ae5d7a7396b36684', 'request_id': 'req-6d991fd5-5008-431c-b7c7-2203379a48f3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:01.275196 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643f0ddc528d45f1ae5d7a7396b36684', 'request_id': 'req-6d991fd5-5008-431c-b7c7-2203379a48f3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:01.285934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] The token cache is being invalidate because project 643f0ddc528d45f1ae5d7a7396b36684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:46:01.286227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:01.286851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:46:01.288369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1397/2758] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:46:00 2022] DELETE /identity/v3/projects/643f0ddc528d45f1ae5d7a7396b36684 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:46:04.090022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d48ab-5438-446e-9dda-9cd32093e1f0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:46:04.090622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d48ab-5438-446e-9dda-9cd32093e1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:46:04.090622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d48ab-5438-446e-9dda-9cd32093e1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:46:04.112574 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7d9d48ab-5438-446e-9dda-9cd32093e1f0 None None] MFA Rules not processed for user `394c67bcc3864bb2aca2f4faec834029`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:46:04.113817 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7d9d48ab-5438-446e-9dda-9cd32093e1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:46:04.128495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:46:04.171580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1362/2759] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:46:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:46:11.355666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:46:14.580161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=16 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:47:20.464282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:47:20.465709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:47:20.522690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d991fd5-5008-431c-b7c7-2203379a48f3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:47:20.526237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf9be8-0e75-48e9-b4c2-c3d0774a2053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:47:20.526828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf9be8-0e75-48e9-b4c2-c3d0774a2053 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:47:20.527104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf9be8-0e75-48e9-b4c2-c3d0774a2053 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:47:20.527370 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf9be8-0e75-48e9-b4c2-c3d0774a2053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:47:20.529264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-61cf9be8-0e75-48e9-b4c2-c3d0774a2053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:47:20.613796 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf9be8-0e75-48e9-b4c2-c3d0774a2053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:47:20.615238 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf9be8-0e75-48e9-b4c2-c3d0774a2053 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:47:20.635182 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1398/2760] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:47:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:47:30.632572 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=63 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:47:53.767739 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d48ab-5438-446e-9dda-9cd32093e1f0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:47:53.777438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d48ab-5438-446e-9dda-9cd32093e1f0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:47:53.781206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c68739-a2ad-4bc4-ac82-dcb4162d6dac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:47:53.781802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c68739-a2ad-4bc4-ac82-dcb4162d6dac service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:47:53.782084 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c68739-a2ad-4bc4-ac82-dcb4162d6dac service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:47:53.782361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c68739-a2ad-4bc4-ac82-dcb4162d6dac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:47:53.784125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-62c68739-a2ad-4bc4-ac82-dcb4162d6dac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:47:53.797539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c68739-a2ad-4bc4-ac82-dcb4162d6dac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:47:53.798952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c68739-a2ad-4bc4-ac82-dcb4162d6dac service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:47:53.810376 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1363/2761] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:47:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:47:57.165147 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf9be8-0e75-48e9-b4c2-c3d0774a2053 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:47:57.166954 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-61cf9be8-0e75-48e9-b4c2-c3d0774a2053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:47:57.182453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf9be8-0e75-48e9-b4c2-c3d0774a2053 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:47:57.189241 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe1887-1882-4f55-805d-f888b97d42c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:47:57.189869 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe1887-1882-4f55-805d-f888b97d42c3 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:47:57.191102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe1887-1882-4f55-805d-f888b97d42c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:47:57.191102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe1887-1882-4f55-805d-f888b97d42c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:47:57.192424 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-33fe1887-1882-4f55-805d-f888b97d42c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:47:57.206309 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe1887-1882-4f55-805d-f888b97d42c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:47:57.209257 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe1887-1882-4f55-805d-f888b97d42c3 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:47:57.220530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1399/2762] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:47:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:48:01.818679 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e674b-e562-4967-a189-cc3c8ecb4cb8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:48:01.820564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e674b-e562-4967-a189-cc3c8ecb4cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:48:01.820564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e674b-e562-4967-a189-cc3c8ecb4cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:48:01.838857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-814e674b-e562-4967-a189-cc3c8ecb4cb8 None None] MFA Rules not processed for user `990056333138468c99292c3f00605e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:48:01.840240 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-814e674b-e562-4967-a189-cc3c8ecb4cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:48:01.895151 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1364/2763] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:48:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:48:07.214958 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:48:11.888041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=31 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:48:19.902702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe1887-1882-4f55-805d-f888b97d42c3 service nova] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:48:19.904154 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-33fe1887-1882-4f55-805d-f888b97d42c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:48:19.915590 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe1887-1882-4f55-805d-f888b97d42c3 service nova] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:48:19.919155 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dad789-a3af-4c4c-8b0f-62401de4c244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:48:19.919732 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dad789-a3af-4c4c-8b0f-62401de4c244 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:48:19.920006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dad789-a3af-4c4c-8b0f-62401de4c244 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:48:19.920304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dad789-a3af-4c4c-8b0f-62401de4c244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:48:19.921947 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b6dad789-a3af-4c4c-8b0f-62401de4c244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:48:19.983913 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dad789-a3af-4c4c-8b0f-62401de4c244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:48:19.985381 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dad789-a3af-4c4c-8b0f-62401de4c244 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:48:19.995452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1400/2764] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:48:26.916980 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e674b-e562-4967-a189-cc3c8ecb4cb8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:48:26.918540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-814e674b-e562-4967-a189-cc3c8ecb4cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:48:26.933536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e674b-e562-4967-a189-cc3c8ecb4cb8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:48:26.938237 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:48:26.938867 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:48:26.939146 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:48:26.939420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] PATH_INFO: `/v3/users/394c67bcc3864bb2aca2f4faec834029` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:48:26.940750 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=394c67bcc3864bb2aca2f4faec834029)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:48:26.942869 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:48:27.097369 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '394c67bcc3864bb2aca2f4faec834029'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:27.175585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '394c67bcc3864bb2aca2f4faec834029'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:27.188061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '394c67bcc3864bb2aca2f4faec834029'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:27.190841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '394c67bcc3864bb2aca2f4faec834029'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:27.190841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '394c67bcc3864bb2aca2f4faec834029'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:27.201922 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1365/2765] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:48:26 2022] DELETE /identity/v3/users/394c67bcc3864bb2aca2f4faec834029 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:48:27.485722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dad789-a3af-4c4c-8b0f-62401de4c244 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:48:27.496186 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dad789-a3af-4c4c-8b0f-62401de4c244 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:48:27.555552 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:48:27.556211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:48:27.556489 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:48:27.556761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] PATH_INFO: `/v3/projects/6b3cfd456d8748b3a1db990d638340af` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:48:27.557873 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3cfd456d8748b3a1db990d638340af)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:48:27.559762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:48:27.672812 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3cfd456d8748b3a1db990d638340af', 'request_id': 'req-6306a0af-4e1e-492b-875e-7f167659ee45'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:27.679146 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3cfd456d8748b3a1db990d638340af', 'request_id': 'req-6306a0af-4e1e-492b-875e-7f167659ee45'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:27.746672 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3cfd456d8748b3a1db990d638340af', 'request_id': 'req-6306a0af-4e1e-492b-875e-7f167659ee45'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:27.758266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] The token cache is being invalidate because project 6b3cfd456d8748b3a1db990d638340af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:48:27.758588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:27.759225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:27.760774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1401/2766] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:48:27 2022] DELETE /identity/v3/projects/6b3cfd456d8748b3a1db990d638340af => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:48:27.765134 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:48:27.766614 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:48:27.780603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d33bc-3ec5-45b3-8bfd-0dce58c66669 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:48:27.835796 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:48:27.836439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:48:27.836715 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:48:27.836985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] PATH_INFO: `/v3/users/2aac329bbae0423faabd3fc4601c859d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:48:27.838105 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aac329bbae0423faabd3fc4601c859d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:48:27.839997 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:48:28.012443 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aac329bbae0423faabd3fc4601c859d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:28.080694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aac329bbae0423faabd3fc4601c859d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:28.093023 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aac329bbae0423faabd3fc4601c859d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:28.093685 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aac329bbae0423faabd3fc4601c859d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:28.094296 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aac329bbae0423faabd3fc4601c859d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:28.096290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:48:28.106969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1366/2767] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:48:27 2022] DELETE /identity/v3/users/2aac329bbae0423faabd3fc4601c859d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:48:28.379897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:48:28.381422 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:48:28.395399 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306a0af-4e1e-492b-875e-7f167659ee45 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:48:28.417922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:48:28.453847 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:48:28.454480 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:48:28.454759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:48:28.455034 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] PATH_INFO: `/v3/projects/4842e6e3e892400cbccac28d678c8190` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:48:28.456249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4842e6e3e892400cbccac28d678c8190)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:48:28.458158 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:48:28.577293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4842e6e3e892400cbccac28d678c8190', 'request_id': 'req-c7e01e9c-b5b7-4310-91cd-feafdca4c529'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:28.583457 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4842e6e3e892400cbccac28d678c8190', 'request_id': 'req-c7e01e9c-b5b7-4310-91cd-feafdca4c529'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:28.629110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4842e6e3e892400cbccac28d678c8190', 'request_id': 'req-c7e01e9c-b5b7-4310-91cd-feafdca4c529'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:28.639625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] The token cache is being invalidate because project 4842e6e3e892400cbccac28d678c8190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:48:28.639921 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:28.640578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:48:28.642051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1402/2768] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:48:28 2022] DELETE /identity/v3/projects/4842e6e3e892400cbccac28d678c8190 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:48:38.349206 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:48:38.636416 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:49:20.394603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:20.396166 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:20.474510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d88913-e761-4ddb-ba1b-3ecdec090cc6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:20.547647 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bab356-c6d9-4f68-9c5c-9f0f154e9821 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:20.548339 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bab356-c6d9-4f68-9c5c-9f0f154e9821 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:20.548899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bab356-c6d9-4f68-9c5c-9f0f154e9821 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:20.548899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bab356-c6d9-4f68-9c5c-9f0f154e9821 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:20.550906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-59bab356-c6d9-4f68-9c5c-9f0f154e9821 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:20.565236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bab356-c6d9-4f68-9c5c-9f0f154e9821 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:20.566788 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bab356-c6d9-4f68-9c5c-9f0f154e9821 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:20.579878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1367/2769] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:49:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:49:30.574889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=62 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:49:30.606781 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:30.608503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:30.675157 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e01e9c-b5b7-4310-91cd-feafdca4c529 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:30.679642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb8adf-7f59-4824-9226-fb29dc6e3b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:30.680300 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb8adf-7f59-4824-9226-fb29dc6e3b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:30.681114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb8adf-7f59-4824-9226-fb29dc6e3b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:30.681526 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb8adf-7f59-4824-9226-fb29dc6e3b5f admin admin] PATH_INFO: `/v3/users/990056333138468c99292c3f00605e6c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:30.682666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb8adf-7f59-4824-9226-fb29dc6e3b5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=990056333138468c99292c3f00605e6c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:30.684625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb8adf-7f59-4824-9226-fb29dc6e3b5f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:30.867509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-00bb8adf-7f59-4824-9226-fb29dc6e3b5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '990056333138468c99292c3f00605e6c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:49:30.940444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-00bb8adf-7f59-4824-9226-fb29dc6e3b5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '990056333138468c99292c3f00605e6c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:49:30.953436 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-00bb8adf-7f59-4824-9226-fb29dc6e3b5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '990056333138468c99292c3f00605e6c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:49:30.960365 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-00bb8adf-7f59-4824-9226-fb29dc6e3b5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '990056333138468c99292c3f00605e6c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:49:30.963143 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-00bb8adf-7f59-4824-9226-fb29dc6e3b5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '990056333138468c99292c3f00605e6c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:49:30.967051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1403/2770] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:49:30 2022] DELETE /identity/v3/users/990056333138468c99292c3f00605e6c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:49:31.199219 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bab356-c6d9-4f68-9c5c-9f0f154e9821 service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:31.210084 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bab356-c6d9-4f68-9c5c-9f0f154e9821 service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:31.267986 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:31.268671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:31.268952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:31.269226 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] PATH_INFO: `/v3/projects/dd66028a147f42e1aa29abd7c22d00f9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:31.270371 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd66028a147f42e1aa29abd7c22d00f9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:31.272307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:31.375963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd66028a147f42e1aa29abd7c22d00f9', 'request_id': 'req-c70e128c-731b-409c-84bf-01e75e4d98c5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:49:31.383396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd66028a147f42e1aa29abd7c22d00f9', 'request_id': 'req-c70e128c-731b-409c-84bf-01e75e4d98c5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:49:31.449074 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd66028a147f42e1aa29abd7c22d00f9', 'request_id': 'req-c70e128c-731b-409c-84bf-01e75e4d98c5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:49:31.459969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] The token cache is being invalidate because project dd66028a147f42e1aa29abd7c22d00f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:49:31.460330 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:49:31.460959 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c70e128c-731b-409c-84bf-01e75e4d98c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:49:31.463562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1368/2771] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:49:31 2022] DELETE /identity/v3/projects/dd66028a147f42e1aa29abd7c22d00f9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:49:31.468654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921539-1af8-4482-ab9f-612c5f93c1f2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:31.468944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921539-1af8-4482-ab9f-612c5f93c1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:31.469219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921539-1af8-4482-ab9f-612c5f93c1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:31.518849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ce921539-1af8-4482-ab9f-612c5f93c1f2 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:49:31.520097 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ce921539-1af8-4482-ab9f-612c5f93c1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:31.567731 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:49:31.578540 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1404/2772] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:49:31.586216 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aba7e0-497d-4e44-95d5-2fce730c45e1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:31.586701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aba7e0-497d-4e44-95d5-2fce730c45e1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:31.587373 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aba7e0-497d-4e44-95d5-2fce730c45e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:31.607067 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-93aba7e0-497d-4e44-95d5-2fce730c45e1 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:49:31.608298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-93aba7e0-497d-4e44-95d5-2fce730c45e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:31.616240 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1369/2773] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:49:31.620426 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce921539-1af8-4482-ab9f-612c5f93c1f2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:31.631639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce921539-1af8-4482-ab9f-612c5f93c1f2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:31.636249 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ff273-d121-41c6-b8af-ac7a5ed66bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:31.638451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ff273-d121-41c6-b8af-ac7a5ed66bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:31.638451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ff273-d121-41c6-b8af-ac7a5ed66bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:31.638451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ff273-d121-41c6-b8af-ac7a5ed66bf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:31.638451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ff273-d121-41c6-b8af-ac7a5ed66bf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:49:31.638451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ff273-d121-41c6-b8af-ac7a5ed66bf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:31.640189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ff273-d121-41c6-b8af-ac7a5ed66bf1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:31.652849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1405/2774] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:49:31.657244 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aba7e0-497d-4e44-95d5-2fce730c45e1 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:31.666966 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aba7e0-497d-4e44-95d5-2fce730c45e1 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:31.671629 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21879e-c4dc-4fe0-878f-3306fcef99cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:31.672258 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21879e-c4dc-4fe0-878f-3306fcef99cb admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:31.672538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21879e-c4dc-4fe0-878f-3306fcef99cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:31.672812 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21879e-c4dc-4fe0-878f-3306fcef99cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:31.673434 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21879e-c4dc-4fe0-878f-3306fcef99cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:31.675331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21879e-c4dc-4fe0-878f-3306fcef99cb admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:31.767148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1370/2775] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:49:31 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:49:31.771658 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ff273-d121-41c6-b8af-ac7a5ed66bf1 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:31.782722 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ff273-d121-41c6-b8af-ac7a5ed66bf1 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:31.840289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334807e1-c28e-422e-a4d9-38a0b16a8f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:31.840884 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334807e1-c28e-422e-a4d9-38a0b16a8f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:31.841159 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334807e1-c28e-422e-a4d9-38a0b16a8f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:31.841430 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334807e1-c28e-422e-a4d9-38a0b16a8f2b admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:31.842041 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334807e1-c28e-422e-a4d9-38a0b16a8f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:31.843940 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334807e1-c28e-422e-a4d9-38a0b16a8f2b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:31.940236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1406/2776] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:49:31 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:49:31.941696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21879e-c4dc-4fe0-878f-3306fcef99cb admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:31.952880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21879e-c4dc-4fe0-878f-3306fcef99cb admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:31.957480 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736b411-579d-447c-9ec3-301e10483a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:31.958073 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736b411-579d-447c-9ec3-301e10483a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:31.958349 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736b411-579d-447c-9ec3-301e10483a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:31.958622 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736b411-579d-447c-9ec3-301e10483a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:31.959261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736b411-579d-447c-9ec3-301e10483a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:31.960906 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736b411-579d-447c-9ec3-301e10483a38 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:31.971163 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1371/2777] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:49:31.975061 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334807e1-c28e-422e-a4d9-38a0b16a8f2b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:31.985173 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334807e1-c28e-422e-a4d9-38a0b16a8f2b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:31.989809 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8ee70-38ea-4245-95a9-b9f0a64204f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:31.990400 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8ee70-38ea-4245-95a9-b9f0a64204f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:31.990678 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8ee70-38ea-4245-95a9-b9f0a64204f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:31.990951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8ee70-38ea-4245-95a9-b9f0a64204f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:31.991595 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8ee70-38ea-4245-95a9-b9f0a64204f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:31.993224 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8ee70-38ea-4245-95a9-b9f0a64204f8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:32.002997 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1407/2778] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:49:32.007224 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736b411-579d-447c-9ec3-301e10483a38 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:32.018353 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736b411-579d-447c-9ec3-301e10483a38 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:32.022917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd729861-77fa-4d5a-a0d8-3ad2958b605b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:32.023554 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd729861-77fa-4d5a-a0d8-3ad2958b605b admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:32.023838 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd729861-77fa-4d5a-a0d8-3ad2958b605b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:32.024145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd729861-77fa-4d5a-a0d8-3ad2958b605b admin admin] PATH_INFO: `/v3/projects/4571d38eeff749fcb073842d5ad4fc65/users/13c06492c8db44759185686bd0a11a57/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:32.039696 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd729861-77fa-4d5a-a0d8-3ad2958b605b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4571d38eeff749fcb073842d5ad4fc65, user_id=13c06492c8db44759185686bd0a11a57, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:32.042561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd729861-77fa-4d5a-a0d8-3ad2958b605b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:32.147124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1372/2779] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:49:32 2022] PUT /identity/v3/projects/4571d38eeff749fcb073842d5ad4fc65/users/13c06492c8db44759185686bd0a11a57/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:49:32.152985 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd2572-526c-4291-95dc-1dd7ccf041fc None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:32.153274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd2572-526c-4291-95dc-1dd7ccf041fc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:32.153548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd2572-526c-4291-95dc-1dd7ccf041fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:32.172910 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4afd2572-526c-4291-95dc-1dd7ccf041fc None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:49:32.174120 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4afd2572-526c-4291-95dc-1dd7ccf041fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:32.233543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1408/2780] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:49:32.244537 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd729861-77fa-4d5a-a0d8-3ad2958b605b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:32.246395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-dd729861-77fa-4d5a-a0d8-3ad2958b605b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:32.268609 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:49:32.306606 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd729861-77fa-4d5a-a0d8-3ad2958b605b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:32.310501 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36084ef2-0094-4716-81fe-0a8184350bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:32.311910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36084ef2-0094-4716-81fe-0a8184350bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:32.311910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36084ef2-0094-4716-81fe-0a8184350bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:32.311910 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36084ef2-0094-4716-81fe-0a8184350bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:32.320625 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36084ef2-0094-4716-81fe-0a8184350bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:32.322756 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36084ef2-0094-4716-81fe-0a8184350bf1 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:32.332312 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1373/2781] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:49:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:49:37.460574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6072417-f574-42e4-886f-dbd0478e2f51 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:37.460574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6072417-f574-42e4-886f-dbd0478e2f51 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:37.460574 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6072417-f574-42e4-886f-dbd0478e2f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:37.491399 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f6072417-f574-42e4-886f-dbd0478e2f51 None None] MFA Rules not processed for user `13c06492c8db44759185686bd0a11a57`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:49:37.492762 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f6072417-f574-42e4-886f-dbd0478e2f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:37.552591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:49:37.576772 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1409/2782] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:49:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:49:37.591983 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36084ef2-0094-4716-81fe-0a8184350bf1 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:37.594107 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-36084ef2-0094-4716-81fe-0a8184350bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:37.686436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36084ef2-0094-4716-81fe-0a8184350bf1 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:37.692807 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f939336-a2bd-49d5-976b-0d1e4ed7468d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:37.693552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f939336-a2bd-49d5-976b-0d1e4ed7468d service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:37.693902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f939336-a2bd-49d5-976b-0d1e4ed7468d service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:37.694296 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f939336-a2bd-49d5-976b-0d1e4ed7468d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:37.705069 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f939336-a2bd-49d5-976b-0d1e4ed7468d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:37.706770 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f939336-a2bd-49d5-976b-0d1e4ed7468d service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:37.718934 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1374/2783] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:49:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:49:40.038966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e15ca-3203-4b96-84a4-cb964a75f872 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:40.039484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e15ca-3203-4b96-84a4-cb964a75f872 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:40.039484 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e15ca-3203-4b96-84a4-cb964a75f872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:40.060423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b08e15ca-3203-4b96-84a4-cb964a75f872 None None] MFA Rules not processed for user `13c06492c8db44759185686bd0a11a57`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:49:40.061760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b08e15ca-3203-4b96-84a4-cb964a75f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:40.069244 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1410/2784] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:49:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:49:40.424996 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f939336-a2bd-49d5-976b-0d1e4ed7468d service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:40.428405 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7f939336-a2bd-49d5-976b-0d1e4ed7468d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:40.439536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f939336-a2bd-49d5-976b-0d1e4ed7468d service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:40.443454 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e5e30-601d-48ff-a592-3258f07bebff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:40.444083 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e5e30-601d-48ff-a592-3258f07bebff service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:40.444368 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e5e30-601d-48ff-a592-3258f07bebff service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:40.444639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e5e30-601d-48ff-a592-3258f07bebff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:40.447210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d54e5e30-601d-48ff-a592-3258f07bebff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:40.458740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e5e30-601d-48ff-a592-3258f07bebff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:40.460741 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e5e30-601d-48ff-a592-3258f07bebff service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:40.470688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1375/2785] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:49:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:49:41.158125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e15ca-3203-4b96-84a4-cb964a75f872 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:41.159651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b08e15ca-3203-4b96-84a4-cb964a75f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:41.233181 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e15ca-3203-4b96-84a4-cb964a75f872 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:41.237982 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06904977-063c-44ef-9d3f-900095d5aeda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:41.238578 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06904977-063c-44ef-9d3f-900095d5aeda service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:41.238854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06904977-063c-44ef-9d3f-900095d5aeda service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:41.239129 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06904977-063c-44ef-9d3f-900095d5aeda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:41.240915 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-06904977-063c-44ef-9d3f-900095d5aeda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:41.252444 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06904977-063c-44ef-9d3f-900095d5aeda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:41.254276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06904977-063c-44ef-9d3f-900095d5aeda service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:41.264742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1411/2786] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:49:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:49:44.506345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e5e30-601d-48ff-a592-3258f07bebff service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:44.517723 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e5e30-601d-48ff-a592-3258f07bebff service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:44.521681 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea08743-b027-4b83-821d-4d8f5af8a99e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:44.522288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea08743-b027-4b83-821d-4d8f5af8a99e service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:44.522567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea08743-b027-4b83-821d-4d8f5af8a99e service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:44.522841 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea08743-b027-4b83-821d-4d8f5af8a99e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:44.525274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3ea08743-b027-4b83-821d-4d8f5af8a99e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:44.535803 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea08743-b027-4b83-821d-4d8f5af8a99e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:44.537431 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea08743-b027-4b83-821d-4d8f5af8a99e service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:44.546675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1376/2787] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:49:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:49:51.260052 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=39 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:49:51.762297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06904977-063c-44ef-9d3f-900095d5aeda service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:49:51.763922 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-06904977-063c-44ef-9d3f-900095d5aeda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:51.780003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06904977-063c-44ef-9d3f-900095d5aeda service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:49:51.786677 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb838e0c-16dd-4b8f-b359-f01c721a59ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:49:51.787256 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb838e0c-16dd-4b8f-b359-f01c721a59ef service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:49:51.787543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb838e0c-16dd-4b8f-b359-f01c721a59ef service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:49:51.787815 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb838e0c-16dd-4b8f-b359-f01c721a59ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:49:51.789710 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-eb838e0c-16dd-4b8f-b359-f01c721a59ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:49:51.801849 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb838e0c-16dd-4b8f-b359-f01c721a59ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:49:51.803421 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb838e0c-16dd-4b8f-b359-f01c721a59ef service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:49:51.813941 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1412/2788] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:49:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:49:54.542776 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=64 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:50:01.809477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:50:39.738708 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea08743-b027-4b83-821d-4d8f5af8a99e service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:50:39.752835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea08743-b027-4b83-821d-4d8f5af8a99e service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:50:39.757787 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c3f90-f0b4-4abf-84de-4741f3b1e9d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:50:39.758375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c3f90-f0b4-4abf-84de-4741f3b1e9d8 service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:50:39.758651 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c3f90-f0b4-4abf-84de-4741f3b1e9d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:50:39.758950 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c3f90-f0b4-4abf-84de-4741f3b1e9d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:50:39.760675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e02c3f90-f0b4-4abf-84de-4741f3b1e9d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:50:39.773145 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c3f90-f0b4-4abf-84de-4741f3b1e9d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:50:39.774694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c3f90-f0b4-4abf-84de-4741f3b1e9d8 service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:50:39.785198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1377/2789] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:50:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:50:44.019834 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5220cb-895a-4b80-993d-0488b9ef55cd None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:50:44.020413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5220cb-895a-4b80-993d-0488b9ef55cd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:50:44.020413 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5220cb-895a-4b80-993d-0488b9ef55cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:50:44.039392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ee5220cb-895a-4b80-993d-0488b9ef55cd None None] MFA Rules not processed for user `13c06492c8db44759185686bd0a11a57`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:50:44.040621 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ee5220cb-895a-4b80-993d-0488b9ef55cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:50:44.047975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1413/2790] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:50:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:50:44.355121 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c3f90-f0b4-4abf-84de-4741f3b1e9d8 service placement] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:50:44.367436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c3f90-f0b4-4abf-84de-4741f3b1e9d8 service placement] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:50:44.371356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaed81c-22bf-4332-a771-80ed7d2c406f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:50:44.371955 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaed81c-22bf-4332-a771-80ed7d2c406f service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:50:44.372295 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaed81c-22bf-4332-a771-80ed7d2c406f service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:50:44.372564 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaed81c-22bf-4332-a771-80ed7d2c406f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:50:44.374397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1eaed81c-22bf-4332-a771-80ed7d2c406f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:50:44.385263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaed81c-22bf-4332-a771-80ed7d2c406f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:50:44.386672 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaed81c-22bf-4332-a771-80ed7d2c406f service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:50:44.395720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1378/2791] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:50:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:50:54.034694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:50:54.392389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=12 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:51:54.174073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c19978-b683-4d61-8fea-bb88201e4b5c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:51:54.174629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c19978-b683-4d61-8fea-bb88201e4b5c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:51:54.174629 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c19978-b683-4d61-8fea-bb88201e4b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:51:54.193344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-47c19978-b683-4d61-8fea-bb88201e4b5c None None] MFA Rules not processed for user `13c06492c8db44759185686bd0a11a57`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:51:54.194549 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-47c19978-b683-4d61-8fea-bb88201e4b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:51:54.201362 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1414/2792] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:51:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:52:04.188836 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:52:45.518692 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaed81c-22bf-4332-a771-80ed7d2c406f service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:52:45.520205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1eaed81c-22bf-4332-a771-80ed7d2c406f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:52:45.580113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaed81c-22bf-4332-a771-80ed7d2c406f service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:52:45.583708 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a55d84-8d02-4646-9ed3-283d597951d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:52:45.584336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a55d84-8d02-4646-9ed3-283d597951d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:52:45.584616 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a55d84-8d02-4646-9ed3-283d597951d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:52:45.584889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a55d84-8d02-4646-9ed3-283d597951d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:52:45.586612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-91a55d84-8d02-4646-9ed3-283d597951d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:52:45.600529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a55d84-8d02-4646-9ed3-283d597951d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:52:45.601984 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a55d84-8d02-4646-9ed3-283d597951d6 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:52:45.613355 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1379/2793] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:52:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:52:53.973219 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c19978-b683-4d61-8fea-bb88201e4b5c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:52:53.982966 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c19978-b683-4d61-8fea-bb88201e4b5c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:52:53.986998 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62fa4c-bb85-4d6b-85b9-67a1d7da013f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:52:53.987591 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62fa4c-bb85-4d6b-85b9-67a1d7da013f service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:52:53.987866 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62fa4c-bb85-4d6b-85b9-67a1d7da013f service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:52:53.988169 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62fa4c-bb85-4d6b-85b9-67a1d7da013f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:52:53.989867 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0d62fa4c-bb85-4d6b-85b9-67a1d7da013f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:52:54.003883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62fa4c-bb85-4d6b-85b9-67a1d7da013f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:52:54.005323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62fa4c-bb85-4d6b-85b9-67a1d7da013f service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:52:54.016178 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1415/2794] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:52:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:52:55.608217 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=26 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:52:58.585801 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a55d84-8d02-4646-9ed3-283d597951d6 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:52:58.600167 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a55d84-8d02-4646-9ed3-283d597951d6 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:52:58.607594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5b38b-407b-42aa-9c09-f6673e0732c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8effbd0670ef41f59b46feb9f0997554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:52:58.608236 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b38b-407b-42aa-9c09-f6673e0732c7 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:52:58.608516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b38b-407b-42aa-9c09-f6673e0732c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:52:58.608789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b38b-407b-42aa-9c09-f6673e0732c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:52:58.610612 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c6c5b38b-407b-42aa-9c09-f6673e0732c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:52:58.625009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5b38b-407b-42aa-9c09-f6673e0732c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:52:58.626610 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5b38b-407b-42aa-9c09-f6673e0732c7 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:52:58.640823 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1380/2795] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:52:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:53:03.606258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06801615-853b-421a-bdc3-7a893dc195e8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:53:03.606768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06801615-853b-421a-bdc3-7a893dc195e8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:53:03.606768 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06801615-853b-421a-bdc3-7a893dc195e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:53:03.625687 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-06801615-853b-421a-bdc3-7a893dc195e8 None None] MFA Rules not processed for user `13c06492c8db44759185686bd0a11a57`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:53:03.626933 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-06801615-853b-421a-bdc3-7a893dc195e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:53:03.634463 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1416/2796] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:53:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:53:08.636248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=6 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:53:13.621059 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=9 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:17.012378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5b38b-407b-42aa-9c09-f6673e0732c7 service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:17.024056 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5b38b-407b-42aa-9c09-f6673e0732c7 service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:17.029786 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ea5bd-061b-4090-927c-a88430c0b766 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:17.030377 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea5bd-061b-4090-927c-a88430c0b766 service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:17.030676 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea5bd-061b-4090-927c-a88430c0b766 service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:17.030957 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea5bd-061b-4090-927c-a88430c0b766 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:17.032742 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b70ea5bd-061b-4090-927c-a88430c0b766 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:17.047232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ea5bd-061b-4090-927c-a88430c0b766 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:17.048992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ea5bd-061b-4090-927c-a88430c0b766 service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:17.059508 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1381/2797] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:54:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:54:17.133467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06801615-853b-421a-bdc3-7a893dc195e8 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:17.142934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06801615-853b-421a-bdc3-7a893dc195e8 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:17.147481 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd3d50-3ba8-4006-9cf4-b2348748d826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:17.148140 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd3d50-3ba8-4006-9cf4-b2348748d826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:17.148421 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd3d50-3ba8-4006-9cf4-b2348748d826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:17.148697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd3d50-3ba8-4006-9cf4-b2348748d826 admin admin] PATH_INFO: `/v3/users/13c06492c8db44759185686bd0a11a57` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:17.149818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd3d50-3ba8-4006-9cf4-b2348748d826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13c06492c8db44759185686bd0a11a57)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:17.152108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd3d50-3ba8-4006-9cf4-b2348748d826 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:17.343394 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dfbd3d50-3ba8-4006-9cf4-b2348748d826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13c06492c8db44759185686bd0a11a57'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:17.403706 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dfbd3d50-3ba8-4006-9cf4-b2348748d826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13c06492c8db44759185686bd0a11a57'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:17.416123 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dfbd3d50-3ba8-4006-9cf4-b2348748d826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13c06492c8db44759185686bd0a11a57'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:17.423543 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dfbd3d50-3ba8-4006-9cf4-b2348748d826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13c06492c8db44759185686bd0a11a57'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:17.424305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dfbd3d50-3ba8-4006-9cf4-b2348748d826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13c06492c8db44759185686bd0a11a57'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:17.430188 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1417/2798] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:54:17 2022] DELETE /identity/v3/users/13c06492c8db44759185686bd0a11a57 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:17.667804 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ea5bd-061b-4090-927c-a88430c0b766 service placement] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:17.678198 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ea5bd-061b-4090-927c-a88430c0b766 service placement] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:17.735065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:17.735688 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:17.735963 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:17.736366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] PATH_INFO: `/v3/projects/4571d38eeff749fcb073842d5ad4fc65` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:17.737532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4571d38eeff749fcb073842d5ad4fc65)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:17.739532 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:17.871022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4571d38eeff749fcb073842d5ad4fc65', 'request_id': 'req-6925ab3e-77c5-4558-9b4e-df1397226ed6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:17.877125 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4571d38eeff749fcb073842d5ad4fc65', 'request_id': 'req-6925ab3e-77c5-4558-9b4e-df1397226ed6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:17.939852 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4571d38eeff749fcb073842d5ad4fc65', 'request_id': 'req-6925ab3e-77c5-4558-9b4e-df1397226ed6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:17.950847 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] The token cache is being invalidate because project 4571d38eeff749fcb073842d5ad4fc65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:54:17.951136 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:17.951755 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6925ab3e-77c5-4558-9b4e-df1397226ed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:17.953272 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1382/2799] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:54:17 2022] DELETE /identity/v3/projects/4571d38eeff749fcb073842d5ad4fc65 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:17.958272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b490dda-d564-4652-9c5c-567a57f3d7b6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:17.958559 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b490dda-d564-4652-9c5c-567a57f3d7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:17.958837 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b490dda-d564-4652-9c5c-567a57f3d7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:17.978312 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9b490dda-d564-4652-9c5c-567a57f3d7b6 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:17.984282 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9b490dda-d564-4652-9c5c-567a57f3d7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:18.045522 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1418/2800] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 22:54:18.054600 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b35b-751b-4413-9a3a-36191c127816 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:18.055109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b35b-751b-4413-9a3a-36191c127816 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:18.055214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b35b-751b-4413-9a3a-36191c127816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:18.075406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f242b35b-751b-4413-9a3a-36191c127816 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:18.076633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f242b35b-751b-4413-9a3a-36191c127816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:18.084325 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1383/2801] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:18.088712 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b490dda-d564-4652-9c5c-567a57f3d7b6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:18.098316 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b490dda-d564-4652-9c5c-567a57f3d7b6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:18.102961 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002f7aa-bab6-4809-b0cd-23376ef85dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:18.103600 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002f7aa-bab6-4809-b0cd-23376ef85dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:18.103875 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002f7aa-bab6-4809-b0cd-23376ef85dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:18.104273 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002f7aa-bab6-4809-b0cd-23376ef85dbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:18.105039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002f7aa-bab6-4809-b0cd-23376ef85dbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:54:18.105344 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002f7aa-bab6-4809-b0cd-23376ef85dbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:18.106926 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002f7aa-bab6-4809-b0cd-23376ef85dbe admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:18.118248 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1419/2802] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:54:18.122972 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242b35b-751b-4413-9a3a-36191c127816 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:18.131935 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242b35b-751b-4413-9a3a-36191c127816 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:18.137439 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ba715-51ee-46cc-8e13-7e31a8062726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:18.138019 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ba715-51ee-46cc-8e13-7e31a8062726 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:18.138294 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ba715-51ee-46cc-8e13-7e31a8062726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:18.138563 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ba715-51ee-46cc-8e13-7e31a8062726 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:18.139178 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ba715-51ee-46cc-8e13-7e31a8062726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:18.141396 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ba715-51ee-46cc-8e13-7e31a8062726 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:18.342065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1384/2803] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:54:18 2022] POST /identity/v3/projects => generated 381 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 22:54:18.345967 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002f7aa-bab6-4809-b0cd-23376ef85dbe admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:18.356124 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002f7aa-bab6-4809-b0cd-23376ef85dbe admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:18.412487 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356ee2b-082f-4bb0-ba3f-2bc81e2d1290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:18.413092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356ee2b-082f-4bb0-ba3f-2bc81e2d1290 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:18.413371 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356ee2b-082f-4bb0-ba3f-2bc81e2d1290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:18.413643 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356ee2b-082f-4bb0-ba3f-2bc81e2d1290 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:18.414268 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9356ee2b-082f-4bb0-ba3f-2bc81e2d1290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:18.416214 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9356ee2b-082f-4bb0-ba3f-2bc81e2d1290 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:18.423933 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:18.518301 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1420/2804] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:54:18 2022] POST /identity/v3/users => generated 291 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:18.522392 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ba715-51ee-46cc-8e13-7e31a8062726 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:18.532990 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ba715-51ee-46cc-8e13-7e31a8062726 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:18.538571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632a416-dae1-4b9b-9889-d182b8bb87c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:18.538571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632a416-dae1-4b9b-9889-d182b8bb87c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:18.538571 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632a416-dae1-4b9b-9889-d182b8bb87c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:18.538917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632a416-dae1-4b9b-9889-d182b8bb87c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:18.539308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632a416-dae1-4b9b-9889-d182b8bb87c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:18.540912 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632a416-dae1-4b9b-9889-d182b8bb87c4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:18.551343 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1385/2805] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:18.555150 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356ee2b-082f-4bb0-ba3f-2bc81e2d1290 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:18.566153 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356ee2b-082f-4bb0-ba3f-2bc81e2d1290 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:18.570614 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd61c35-364a-4127-af88-1de844da8496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:18.572886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd61c35-364a-4127-af88-1de844da8496 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:18.572886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd61c35-364a-4127-af88-1de844da8496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:18.572886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd61c35-364a-4127-af88-1de844da8496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:18.572886 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd61c35-364a-4127-af88-1de844da8496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:18.574247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd61c35-364a-4127-af88-1de844da8496 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:18.584505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1421/2806] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:18.588597 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632a416-dae1-4b9b-9889-d182b8bb87c4 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:18.598540 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632a416-dae1-4b9b-9889-d182b8bb87c4 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:18.603015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ad16e-d443-4db8-b4ca-3b837568084a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:18.603645 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad16e-d443-4db8-b4ca-3b837568084a admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:18.604497 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad16e-d443-4db8-b4ca-3b837568084a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:18.604808 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad16e-d443-4db8-b4ca-3b837568084a admin admin] PATH_INFO: `/v3/projects/9b3b6c7a36c044169ef557330b58006f/users/8aa7b9d886c44938865818cb444d2e69/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:18.619058 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ad16e-d443-4db8-b4ca-3b837568084a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b3b6c7a36c044169ef557330b58006f, user_id=8aa7b9d886c44938865818cb444d2e69, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:18.622498 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ad16e-d443-4db8-b4ca-3b837568084a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:18.699553 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1386/2807] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:18 2022] PUT /identity/v3/projects/9b3b6c7a36c044169ef557330b58006f/users/8aa7b9d886c44938865818cb444d2e69/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:18.761275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8eb40-0e14-4376-9fd8-df78cc2f1835 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:18.761639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8eb40-0e14-4376-9fd8-df78cc2f1835 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:18.761827 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8eb40-0e14-4376-9fd8-df78cc2f1835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:18.789339 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c6f8eb40-0e14-4376-9fd8-df78cc2f1835 None None] MFA Rules not processed for user `8aa7b9d886c44938865818cb444d2e69`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:18.790558 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c6f8eb40-0e14-4376-9fd8-df78cc2f1835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:18.853163 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1422/2808] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:18.858313 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ad16e-d443-4db8-b4ca-3b837568084a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:18.871603 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:18.920461 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ad16e-d443-4db8-b4ca-3b837568084a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:18.925232 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe872e-1f4d-4e98-bc2b-433644267b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:18.925830 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe872e-1f4d-4e98-bc2b-433644267b11 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:18.926109 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe872e-1f4d-4e98-bc2b-433644267b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:18.926384 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe872e-1f4d-4e98-bc2b-433644267b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:18.927008 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe872e-1f4d-4e98-bc2b-433644267b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:18.928976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe872e-1f4d-4e98-bc2b-433644267b11 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:19.046205 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1387/2809] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:54:18 2022] POST /identity/v3/projects => generated 381 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:19.048983 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8eb40-0e14-4376-9fd8-df78cc2f1835 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:19.059745 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8eb40-0e14-4376-9fd8-df78cc2f1835 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:19.115054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aa231-decd-46f0-bec2-3cdcbcfa3bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:19.115585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aa231-decd-46f0-bec2-3cdcbcfa3bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:19.115863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aa231-decd-46f0-bec2-3cdcbcfa3bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:19.116172 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aa231-decd-46f0-bec2-3cdcbcfa3bd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:19.116793 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09aa231-decd-46f0-bec2-3cdcbcfa3bd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:19.118702 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09aa231-decd-46f0-bec2-3cdcbcfa3bd8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:19.202805 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1423/2810] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:54:19 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:19.207218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe872e-1f4d-4e98-bc2b-433644267b11 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:19.218775 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe872e-1f4d-4e98-bc2b-433644267b11 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:19.223366 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644dbd35-d4f3-4edb-8ce8-b657699ecd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:19.223952 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dbd35-d4f3-4edb-8ce8-b657699ecd3a admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:19.224261 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dbd35-d4f3-4edb-8ce8-b657699ecd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:19.224531 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dbd35-d4f3-4edb-8ce8-b657699ecd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:19.225196 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644dbd35-d4f3-4edb-8ce8-b657699ecd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:19.226765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644dbd35-d4f3-4edb-8ce8-b657699ecd3a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:19.237683 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1388/2811] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:19.241707 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aa231-decd-46f0-bec2-3cdcbcfa3bd8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:19.252051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aa231-decd-46f0-bec2-3cdcbcfa3bd8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:19.256717 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f04c92-ba74-461c-83d3-ce68e3954659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:19.257361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f04c92-ba74-461c-83d3-ce68e3954659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:19.257639 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f04c92-ba74-461c-83d3-ce68e3954659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:19.257914 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f04c92-ba74-461c-83d3-ce68e3954659 admin admin] PATH_INFO: `/v3/projects/b7d810061ef345bb8525d812074fe884/users/c8f32ab7f283463a95e9d74008d24cd4/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:19.273625 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f04c92-ba74-461c-83d3-ce68e3954659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d810061ef345bb8525d812074fe884, user_id=c8f32ab7f283463a95e9d74008d24cd4, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:19.277108 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f04c92-ba74-461c-83d3-ce68e3954659 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:19.368585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1424/2812] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:19 2022] PUT /identity/v3/projects/b7d810061ef345bb8525d812074fe884/users/c8f32ab7f283463a95e9d74008d24cd4/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:19.372847 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644dbd35-d4f3-4edb-8ce8-b657699ecd3a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:19.383013 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644dbd35-d4f3-4edb-8ce8-b657699ecd3a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:19.439033 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ed790-01dc-4616-8601-a51c022a5561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:19.439628 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ed790-01dc-4616-8601-a51c022a5561 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:19.439905 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ed790-01dc-4616-8601-a51c022a5561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:19.440229 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ed790-01dc-4616-8601-a51c022a5561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:19.440862 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ed790-01dc-4616-8601-a51c022a5561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:19.442432 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ed790-01dc-4616-8601-a51c022a5561 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:19.452428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1389/2813] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:19.456557 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f04c92-ba74-461c-83d3-ce68e3954659 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:19.467647 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f04c92-ba74-461c-83d3-ce68e3954659 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:19.472320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffa28c-801e-4713-9ce1-1b2e17971770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:19.472962 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffa28c-801e-4713-9ce1-1b2e17971770 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:19.473245 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffa28c-801e-4713-9ce1-1b2e17971770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:19.473521 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffa28c-801e-4713-9ce1-1b2e17971770 admin admin] PATH_INFO: `/v3/projects/b7d810061ef345bb8525d812074fe884/users/c8f32ab7f283463a95e9d74008d24cd4/roles/108664305d2546949f7327b65d4cb667` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:19.475462 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffa28c-801e-4713-9ce1-1b2e17971770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d810061ef345bb8525d812074fe884, user_id=c8f32ab7f283463a95e9d74008d24cd4, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:19.478171 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffa28c-801e-4713-9ce1-1b2e17971770 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:19.488340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1425/2814] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:19 2022] PUT /identity/v3/projects/b7d810061ef345bb8525d812074fe884/users/c8f32ab7f283463a95e9d74008d24cd4/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:19.498345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e1665-a51a-45e1-a3ec-812542f04e84 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:19.498637 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e1665-a51a-45e1-a3ec-812542f04e84 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:19.498919 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e1665-a51a-45e1-a3ec-812542f04e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:19.527034 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-167e1665-a51a-45e1-a3ec-812542f04e84 None None] MFA Rules not processed for user `c8f32ab7f283463a95e9d74008d24cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:19.528274 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-167e1665-a51a-45e1-a3ec-812542f04e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:19.595226 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1390/2815] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:19.609019 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffa28c-801e-4713-9ce1-1b2e17971770 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:19.610709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b0ffa28c-801e-4713-9ce1-1b2e17971770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:19.633354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:19.670287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffa28c-801e-4713-9ce1-1b2e17971770 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:19.673965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14585ec6-ded9-4b07-a8d1-689027d18898 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:19.674560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14585ec6-ded9-4b07-a8d1-689027d18898 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:19.674840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14585ec6-ded9-4b07-a8d1-689027d18898 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:19.675115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14585ec6-ded9-4b07-a8d1-689027d18898 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:19.684654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14585ec6-ded9-4b07-a8d1-689027d18898 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:19.686080 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14585ec6-ded9-4b07-a8d1-689027d18898 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:19.695432 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1426/2816] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:54:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:54:20.779389 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e1665-a51a-45e1-a3ec-812542f04e84 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:20.788576 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e1665-a51a-45e1-a3ec-812542f04e84 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:20.793154 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba800f-c1fc-4781-a3f1-119f82c40897 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8f32ab7f283463a95e9d74008d24cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7d810061ef345bb8525d812074fe884', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:20.793757 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba800f-c1fc-4781-a3f1-119f82c40897 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:20.794036 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba800f-c1fc-4781-a3f1-119f82c40897 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:20.794310 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba800f-c1fc-4781-a3f1-119f82c40897 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:20.794948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba800f-c1fc-4781-a3f1-119f82c40897 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:54:20.795298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba800f-c1fc-4781-a3f1-119f82c40897 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:20.796849 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba800f-c1fc-4781-a3f1-119f82c40897 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:20.799086 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:20.808360 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1391/2817] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:54:20.812363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14585ec6-ded9-4b07-a8d1-689027d18898 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:20.821318 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14585ec6-ded9-4b07-a8d1-689027d18898 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:20.826455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e508fc-1395-471c-86a6-9fbd1677dee2 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8f32ab7f283463a95e9d74008d24cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7d810061ef345bb8525d812074fe884', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:20.827047 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e508fc-1395-471c-86a6-9fbd1677dee2 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:20.827326 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e508fc-1395-471c-86a6-9fbd1677dee2 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:20.827606 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e508fc-1395-471c-86a6-9fbd1677dee2 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:20.828253 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e508fc-1395-471c-86a6-9fbd1677dee2 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:20.830141 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e508fc-1395-471c-86a6-9fbd1677dee2 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:20.913655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1427/2818] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:54:20 2022] POST /identity/v3/projects => generated 355 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:20.918486 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba800f-c1fc-4781-a3f1-119f82c40897 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:20.928519 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba800f-c1fc-4781-a3f1-119f82c40897 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:20.986430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5afdf-da4b-4136-a8f9-c73581ece4af tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8f32ab7f283463a95e9d74008d24cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7d810061ef345bb8525d812074fe884', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:20.987029 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5afdf-da4b-4136-a8f9-c73581ece4af tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:20.987307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5afdf-da4b-4136-a8f9-c73581ece4af tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:20.987608 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5afdf-da4b-4136-a8f9-c73581ece4af tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:20.988283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5afdf-da4b-4136-a8f9-c73581ece4af tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:54:20.988585 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5afdf-da4b-4136-a8f9-c73581ece4af tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:20.990170 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5afdf-da4b-4136-a8f9-c73581ece4af tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:21.001922 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1392/2819] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 22:54:21.038289 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e508fc-1395-471c-86a6-9fbd1677dee2 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:21.049840 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e508fc-1395-471c-86a6-9fbd1677dee2 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:21.054586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8f32ab7f283463a95e9d74008d24cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7d810061ef345bb8525d812074fe884', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:21.055210 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:21.055493 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:21.055766 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] PATH_INFO: `/v3/projects/194749cdd2154898bc044be0172fe7be` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:21.056929 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: Authorizing `identity:delete_project(project_id=194749cdd2154898bc044be0172fe7be)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:21.059448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:21.169806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194749cdd2154898bc044be0172fe7be', 'request_id': 'req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:21.176539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194749cdd2154898bc044be0172fe7be', 'request_id': 'req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:21.265022 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194749cdd2154898bc044be0172fe7be', 'request_id': 'req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:21.275783 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] The token cache is being invalidate because project 194749cdd2154898bc044be0172fe7be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:54:21.280579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:21.280579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9b9f3efa-dfae-46f1-9b3a-bc6d54fe7ed3 tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:21.280579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1428/2820] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:54:21 2022] DELETE /identity/v3/projects/194749cdd2154898bc044be0172fe7be => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:21.819123 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5afdf-da4b-4136-a8f9-c73581ece4af tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:21.821778 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c6d5afdf-da4b-4136-a8f9-c73581ece4af tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:21.886175 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5afdf-da4b-4136-a8f9-c73581ece4af tempest-VolumeQuotaClassesTest-1134362480 tempest-VolumeQuotaClassesTest-1134362480-project-admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:21.940967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:21.941593 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:21.941991 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:21.942263 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] PATH_INFO: `/v3/users/8aa7b9d886c44938865818cb444d2e69` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:21.943400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aa7b9d886c44938865818cb444d2e69)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:21.945393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:22.101317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:22.109290 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aa7b9d886c44938865818cb444d2e69'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:22.168363 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aa7b9d886c44938865818cb444d2e69'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:22.180475 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aa7b9d886c44938865818cb444d2e69'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:22.181138 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aa7b9d886c44938865818cb444d2e69'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:22.181734 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aa7b9d886c44938865818cb444d2e69'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:22.193736 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1393/2821] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:54:21 2022] DELETE /identity/v3/users/8aa7b9d886c44938865818cb444d2e69 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:22.198442 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4050a93-ad9c-4f38-8a58-af4ca961b64f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:22.198950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4050a93-ad9c-4f38-8a58-af4ca961b64f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:22.198950 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4050a93-ad9c-4f38-8a58-af4ca961b64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:22.218078 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b4050a93-ad9c-4f38-8a58-af4ca961b64f None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:22.219276 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b4050a93-ad9c-4f38-8a58-af4ca961b64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:22.279122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1429/2822] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:22.288758 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:22.290159 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:22.348134 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f223f-d8d8-471d-898d-ca020489f8bf admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:22.351768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd295f-f802-4ad5-b171-72545e99c80a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:22.352393 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd295f-f802-4ad5-b171-72545e99c80a service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:22.352675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd295f-f802-4ad5-b171-72545e99c80a service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:22.352948 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd295f-f802-4ad5-b171-72545e99c80a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:22.361836 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd295f-f802-4ad5-b171-72545e99c80a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:22.363241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd295f-f802-4ad5-b171-72545e99c80a service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:22.372881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1394/2823] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:54:22.766841 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4050a93-ad9c-4f38-8a58-af4ca961b64f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:22.771582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b4050a93-ad9c-4f38-8a58-af4ca961b64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:22.783104 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4050a93-ad9c-4f38-8a58-af4ca961b64f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:22.788728 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:22.789359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:22.789666 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:22.789937 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] PATH_INFO: `/v3/projects/9b3b6c7a36c044169ef557330b58006f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:22.791068 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b3b6c7a36c044169ef557330b58006f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:22.793305 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:22.815991 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:22.948189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b3b6c7a36c044169ef557330b58006f', 'request_id': 'req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:22.954588 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b3b6c7a36c044169ef557330b58006f', 'request_id': 'req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:23.037073 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b3b6c7a36c044169ef557330b58006f', 'request_id': 'req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:23.048311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] The token cache is being invalidate because project 9b3b6c7a36c044169ef557330b58006f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:54:23.048602 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:23.049228 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:23.050744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1430/2824] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:54:22 2022] DELETE /identity/v3/projects/9b3b6c7a36c044169ef557330b58006f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:23.054765 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd295f-f802-4ad5-b171-72545e99c80a service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:23.065009 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd295f-f802-4ad5-b171-72545e99c80a service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:23.120839 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01ffa6-b245-4846-886c-64183a8c102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:23.126026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01ffa6-b245-4846-886c-64183a8c102c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:23.126026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01ffa6-b245-4846-886c-64183a8c102c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:23.126026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01ffa6-b245-4846-886c-64183a8c102c admin admin] PATH_INFO: `/v3/users/c8f32ab7f283463a95e9d74008d24cd4` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:23.126026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01ffa6-b245-4846-886c-64183a8c102c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f32ab7f283463a95e9d74008d24cd4)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:23.126026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01ffa6-b245-4846-886c-64183a8c102c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:23.295616 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0a01ffa6-b245-4846-886c-64183a8c102c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f32ab7f283463a95e9d74008d24cd4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:23.384969 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0a01ffa6-b245-4846-886c-64183a8c102c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f32ab7f283463a95e9d74008d24cd4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:23.391420 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:23.398403 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0a01ffa6-b245-4846-886c-64183a8c102c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f32ab7f283463a95e9d74008d24cd4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:23.400572 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0a01ffa6-b245-4846-886c-64183a8c102c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f32ab7f283463a95e9d74008d24cd4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:23.400572 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0a01ffa6-b245-4846-886c-64183a8c102c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f32ab7f283463a95e9d74008d24cd4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:23.412292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1395/2825] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:54:23 2022] DELETE /identity/v3/users/c8f32ab7f283463a95e9d74008d24cd4 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:23.815016 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:23.816640 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:23.834877 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5148478-8f9f-4c3d-8d8c-cf4d7c7d43c7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:23.893455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:23.894085 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:23.894360 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:23.894628 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] PATH_INFO: `/v3/projects/b7d810061ef345bb8525d812074fe884` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:23.895736 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d810061ef345bb8525d812074fe884)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:23.897697 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:24.020878 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d810061ef345bb8525d812074fe884', 'request_id': 'req-f63b6ff4-016f-43c9-91c2-10ffede5c91e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:24.028035 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d810061ef345bb8525d812074fe884', 'request_id': 'req-f63b6ff4-016f-43c9-91c2-10ffede5c91e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:24.099412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d810061ef345bb8525d812074fe884', 'request_id': 'req-f63b6ff4-016f-43c9-91c2-10ffede5c91e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:24.110443 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] The token cache is being invalidate because project b7d810061ef345bb8525d812074fe884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:54:24.110730 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:24.111359 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b6ff4-016f-43c9-91c2-10ffede5c91e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:24.112854 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1431/2826] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:54:23 2022] DELETE /identity/v3/projects/b7d810061ef345bb8525d812074fe884 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:24.117828 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699ddca-7b4d-4015-942b-1c667c95712b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:24.118385 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699ddca-7b4d-4015-942b-1c667c95712b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:24.118500 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699ddca-7b4d-4015-942b-1c667c95712b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:24.139043 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1699ddca-7b4d-4015-942b-1c667c95712b None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:24.140385 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1699ddca-7b4d-4015-942b-1c667c95712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:24.197347 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1396/2827] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:24.206979 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac36142-fb19-482c-a8ba-a74fdc8403da None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:24.207262 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac36142-fb19-482c-a8ba-a74fdc8403da None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:24.207562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac36142-fb19-482c-a8ba-a74fdc8403da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:24.226777 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cac36142-fb19-482c-a8ba-a74fdc8403da None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:24.227995 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cac36142-fb19-482c-a8ba-a74fdc8403da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:24.235861 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1432/2828] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:24.239988 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699ddca-7b4d-4015-942b-1c667c95712b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:24.250011 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699ddca-7b4d-4015-942b-1c667c95712b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:24.254495 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b581976-0f29-47f4-8200-0fc8735efa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:24.255082 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581976-0f29-47f4-8200-0fc8735efa0f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:24.255359 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581976-0f29-47f4-8200-0fc8735efa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:24.255628 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581976-0f29-47f4-8200-0fc8735efa0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:24.256292 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b581976-0f29-47f4-8200-0fc8735efa0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:54:24.256595 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b581976-0f29-47f4-8200-0fc8735efa0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:24.258169 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b581976-0f29-47f4-8200-0fc8735efa0f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:24.270212 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1397/2829] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:54:24.274405 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac36142-fb19-482c-a8ba-a74fdc8403da None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:24.284096 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac36142-fb19-482c-a8ba-a74fdc8403da None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:24.288681 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ef99c-b059-449d-88e7-c341c33c364b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:24.289269 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ef99c-b059-449d-88e7-c341c33c364b admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:24.289547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ef99c-b059-449d-88e7-c341c33c364b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:24.289818 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ef99c-b059-449d-88e7-c341c33c364b admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:24.290449 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ef99c-b059-449d-88e7-c341c33c364b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:24.292560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ef99c-b059-449d-88e7-c341c33c364b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:24.389409 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1433/2830] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:54:24 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:24.393578 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b581976-0f29-47f4-8200-0fc8735efa0f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:24.403738 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b581976-0f29-47f4-8200-0fc8735efa0f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:24.459515 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70408aad-9f3e-4e06-9cf8-4458431a74b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:24.460131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70408aad-9f3e-4e06-9cf8-4458431a74b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:24.460412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70408aad-9f3e-4e06-9cf8-4458431a74b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:24.460681 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70408aad-9f3e-4e06-9cf8-4458431a74b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:24.461309 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70408aad-9f3e-4e06-9cf8-4458431a74b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:24.463239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70408aad-9f3e-4e06-9cf8-4458431a74b2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:24.547682 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1398/2831] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:54:24 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:24.552006 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ef99c-b059-449d-88e7-c341c33c364b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:24.563192 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ef99c-b059-449d-88e7-c341c33c364b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:24.567862 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af9f1-fb6f-4440-8991-50fff868596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:24.568497 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af9f1-fb6f-4440-8991-50fff868596e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:24.568775 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af9f1-fb6f-4440-8991-50fff868596e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:24.569048 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af9f1-fb6f-4440-8991-50fff868596e admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:24.569709 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37af9f1-fb6f-4440-8991-50fff868596e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:24.571278 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37af9f1-fb6f-4440-8991-50fff868596e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:24.573403 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:24.581604 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1434/2832] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:24.585562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70408aad-9f3e-4e06-9cf8-4458431a74b2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:24.595858 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70408aad-9f3e-4e06-9cf8-4458431a74b2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:24.600618 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2dc11-a7db-4d62-8f4b-ef55c3d36742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:24.601208 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2dc11-a7db-4d62-8f4b-ef55c3d36742 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:24.601496 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2dc11-a7db-4d62-8f4b-ef55c3d36742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:24.601771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2dc11-a7db-4d62-8f4b-ef55c3d36742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:24.602414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2dc11-a7db-4d62-8f4b-ef55c3d36742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:24.604587 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2dc11-a7db-4d62-8f4b-ef55c3d36742 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:24.615104 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1399/2833] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:54:24.620784 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af9f1-fb6f-4440-8991-50fff868596e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:24.629774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af9f1-fb6f-4440-8991-50fff868596e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:24.634418 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde854a6-cd9c-4eb0-9deb-32f028b57add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:24.635192 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde854a6-cd9c-4eb0-9deb-32f028b57add admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:24.635630 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde854a6-cd9c-4eb0-9deb-32f028b57add admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:24.635938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde854a6-cd9c-4eb0-9deb-32f028b57add admin admin] PATH_INFO: `/v3/projects/70a5bebd3c84405897f785c234afc027/users/0baeb14fa44940d5890b241ec230e1ff/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:24.651955 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde854a6-cd9c-4eb0-9deb-32f028b57add admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a5bebd3c84405897f785c234afc027, user_id=0baeb14fa44940d5890b241ec230e1ff, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:24.654685 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde854a6-cd9c-4eb0-9deb-32f028b57add admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:24.747548 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1435/2834] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:24 2022] PUT /identity/v3/projects/70a5bebd3c84405897f785c234afc027/users/0baeb14fa44940d5890b241ec230e1ff/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:24.757767 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb746d-71a9-4ac1-855a-d5650ef92184 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:24.758069 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb746d-71a9-4ac1-855a-d5650ef92184 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:24.758334 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb746d-71a9-4ac1-855a-d5650ef92184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:24.786641 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-fdbb746d-71a9-4ac1-855a-d5650ef92184 None None] MFA Rules not processed for user `0baeb14fa44940d5890b241ec230e1ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:24.787918 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-fdbb746d-71a9-4ac1-855a-d5650ef92184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:24.820737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:24.852992 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1400/2835] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:24.857919 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde854a6-cd9c-4eb0-9deb-32f028b57add admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:24.920107 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde854a6-cd9c-4eb0-9deb-32f028b57add admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:24.924821 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941728f-17ee-4259-8ae4-2f217e913b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:24.925415 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941728f-17ee-4259-8ae4-2f217e913b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:24.925780 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941728f-17ee-4259-8ae4-2f217e913b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:24.926054 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941728f-17ee-4259-8ae4-2f217e913b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:24.926734 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941728f-17ee-4259-8ae4-2f217e913b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:24.928663 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941728f-17ee-4259-8ae4-2f217e913b9a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:25.058797 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1436/2836] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:54:24 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:25.060214 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb746d-71a9-4ac1-855a-d5650ef92184 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:25.071027 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb746d-71a9-4ac1-855a-d5650ef92184 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:25.126412 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57622afb-71c8-44ff-8db5-a1f295edcf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:25.127065 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57622afb-71c8-44ff-8db5-a1f295edcf44 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:25.127280 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57622afb-71c8-44ff-8db5-a1f295edcf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:25.127554 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57622afb-71c8-44ff-8db5-a1f295edcf44 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:25.128202 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57622afb-71c8-44ff-8db5-a1f295edcf44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:25.130099 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57622afb-71c8-44ff-8db5-a1f295edcf44 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:25.233298 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1401/2837] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:54:25 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:25.237204 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941728f-17ee-4259-8ae4-2f217e913b9a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:25.248437 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941728f-17ee-4259-8ae4-2f217e913b9a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:25.253142 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f3400-c7f4-4fe9-8a38-005eff8a94cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:25.254634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f3400-c7f4-4fe9-8a38-005eff8a94cf admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:25.254634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f3400-c7f4-4fe9-8a38-005eff8a94cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:25.254634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f3400-c7f4-4fe9-8a38-005eff8a94cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:25.254984 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f3400-c7f4-4fe9-8a38-005eff8a94cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:25.256620 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f3400-c7f4-4fe9-8a38-005eff8a94cf admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:25.267227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1437/2838] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:25.271402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57622afb-71c8-44ff-8db5-a1f295edcf44 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:25.281307 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57622afb-71c8-44ff-8db5-a1f295edcf44 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:25.285846 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e5317-8436-4f9e-b816-ee86d7c1e582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:25.286428 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e5317-8436-4f9e-b816-ee86d7c1e582 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:25.286706 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e5317-8436-4f9e-b816-ee86d7c1e582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:25.286976 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e5317-8436-4f9e-b816-ee86d7c1e582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:25.287643 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e5317-8436-4f9e-b816-ee86d7c1e582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:25.289444 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e5317-8436-4f9e-b816-ee86d7c1e582 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:25.300210 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1402/2839] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:54:25.304309 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f3400-c7f4-4fe9-8a38-005eff8a94cf admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:25.314933 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f3400-c7f4-4fe9-8a38-005eff8a94cf admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:25.319530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878ce4e-65bb-4820-9357-6d6ed361d14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:25.320201 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878ce4e-65bb-4820-9357-6d6ed361d14f admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:25.320482 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878ce4e-65bb-4820-9357-6d6ed361d14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:25.320759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878ce4e-65bb-4820-9357-6d6ed361d14f admin admin] PATH_INFO: `/v3/projects/a1122c0735bd4109938e3d1cb0cb7db2/users/90f7677a63e5422d926d6deccc46f39c/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:25.335319 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878ce4e-65bb-4820-9357-6d6ed361d14f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1122c0735bd4109938e3d1cb0cb7db2, user_id=90f7677a63e5422d926d6deccc46f39c, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:25.338023 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878ce4e-65bb-4820-9357-6d6ed361d14f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:25.403853 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1438/2840] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:25 2022] PUT /identity/v3/projects/a1122c0735bd4109938e3d1cb0cb7db2/users/90f7677a63e5422d926d6deccc46f39c/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:25.414575 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a63d2-a7a4-4ff6-9659-d697bb0cfa51 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:25.414964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a63d2-a7a4-4ff6-9659-d697bb0cfa51 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:25.415248 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a63d2-a7a4-4ff6-9659-d697bb0cfa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:25.444124 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-bc9a63d2-a7a4-4ff6-9659-d697bb0cfa51 None None] MFA Rules not processed for user `90f7677a63e5422d926d6deccc46f39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:25.445332 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bc9a63d2-a7a4-4ff6-9659-d697bb0cfa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:25.507889 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1403/2841] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:25.512948 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878ce4e-65bb-4820-9357-6d6ed361d14f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:25.574309 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878ce4e-65bb-4820-9357-6d6ed361d14f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:25.578951 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57898545-61ee-41c8-936a-ff89c04a6cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:25.579546 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57898545-61ee-41c8-936a-ff89c04a6cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:25.580533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57898545-61ee-41c8-936a-ff89c04a6cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:25.580533 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57898545-61ee-41c8-936a-ff89c04a6cb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:25.580738 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57898545-61ee-41c8-936a-ff89c04a6cb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:25.582641 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57898545-61ee-41c8-936a-ff89c04a6cb4 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:25.691127 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1439/2842] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:54:25 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:25.695164 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a63d2-a7a4-4ff6-9659-d697bb0cfa51 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:25.706015 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a63d2-a7a4-4ff6-9659-d697bb0cfa51 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:25.747079 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:25.762904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a8aa2-c7a6-455c-8065-c731e2ff8409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:25.763509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a8aa2-c7a6-455c-8065-c731e2ff8409 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:25.763799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a8aa2-c7a6-455c-8065-c731e2ff8409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:25.764111 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a8aa2-c7a6-455c-8065-c731e2ff8409 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:25.764744 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a8aa2-c7a6-455c-8065-c731e2ff8409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:25.766649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a8aa2-c7a6-455c-8065-c731e2ff8409 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:25.842871 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1404/2843] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:54:25 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:25.846939 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57898545-61ee-41c8-936a-ff89c04a6cb4 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:25.858028 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57898545-61ee-41c8-936a-ff89c04a6cb4 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:25.862892 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf23a8-ed37-4bdb-84d6-1c73a723be61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:25.863519 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf23a8-ed37-4bdb-84d6-1c73a723be61 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:25.863798 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf23a8-ed37-4bdb-84d6-1c73a723be61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:25.864110 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf23a8-ed37-4bdb-84d6-1c73a723be61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:25.864759 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf23a8-ed37-4bdb-84d6-1c73a723be61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:25.866352 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf23a8-ed37-4bdb-84d6-1c73a723be61 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:25.868619 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:25.877544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1440/2844] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:25.881693 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a8aa2-c7a6-455c-8065-c731e2ff8409 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:25.892088 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a8aa2-c7a6-455c-8065-c731e2ff8409 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:25.897287 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a6de0-6297-4aaa-bec6-ef7e302b44f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:25.897509 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a6de0-6297-4aaa-bec6-ef7e302b44f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:25.897789 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a6de0-6297-4aaa-bec6-ef7e302b44f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:25.898069 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a6de0-6297-4aaa-bec6-ef7e302b44f9 admin admin] PATH_INFO: `/v3/projects/be3511d14be343948a23281aaa072f96/users/668831ec48d742f0a87720f985cbc722/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:25.913356 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a6de0-6297-4aaa-bec6-ef7e302b44f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3511d14be343948a23281aaa072f96, user_id=668831ec48d742f0a87720f985cbc722, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:25.916094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a6de0-6297-4aaa-bec6-ef7e302b44f9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:25.974426 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1405/2845] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:25 2022] PUT /identity/v3/projects/be3511d14be343948a23281aaa072f96/users/668831ec48d742f0a87720f985cbc722/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 22:54:25.978380 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf23a8-ed37-4bdb-84d6-1c73a723be61 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:25.989161 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf23a8-ed37-4bdb-84d6-1c73a723be61 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:26.047115 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c929a-3a62-4aeb-a249-2d1b9129686e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:26.047642 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c929a-3a62-4aeb-a249-2d1b9129686e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:26.047920 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c929a-3a62-4aeb-a249-2d1b9129686e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:26.048225 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c929a-3a62-4aeb-a249-2d1b9129686e admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:26.048871 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c929a-3a62-4aeb-a249-2d1b9129686e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:26.050470 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c929a-3a62-4aeb-a249-2d1b9129686e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:26.060772 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1441/2846] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:26.064899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a6de0-6297-4aaa-bec6-ef7e302b44f9 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:26.075740 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a6de0-6297-4aaa-bec6-ef7e302b44f9 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:26.080409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fea6bc-8f5b-4793-bb20-33e7cd082544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:26.099880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fea6bc-8f5b-4793-bb20-33e7cd082544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:26.099880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fea6bc-8f5b-4793-bb20-33e7cd082544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:26.099880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fea6bc-8f5b-4793-bb20-33e7cd082544 admin admin] PATH_INFO: `/v3/projects/be3511d14be343948a23281aaa072f96/users/668831ec48d742f0a87720f985cbc722/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:26.099880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fea6bc-8f5b-4793-bb20-33e7cd082544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3511d14be343948a23281aaa072f96, user_id=668831ec48d742f0a87720f985cbc722, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:26.099880 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fea6bc-8f5b-4793-bb20-33e7cd082544 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:26.100406 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1406/2847] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:26 2022] PUT /identity/v3/projects/be3511d14be343948a23281aaa072f96/users/668831ec48d742f0a87720f985cbc722/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:26.110198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd843c0-8520-4b93-a42e-0721168920e6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:26.110489 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd843c0-8520-4b93-a42e-0721168920e6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:26.110801 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd843c0-8520-4b93-a42e-0721168920e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:26.138654 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ddd843c0-8520-4b93-a42e-0721168920e6 None None] MFA Rules not processed for user `668831ec48d742f0a87720f985cbc722`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:26.139870 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ddd843c0-8520-4b93-a42e-0721168920e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:26.209569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1442/2848] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:26.222220 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fea6bc-8f5b-4793-bb20-33e7cd082544 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:26.223964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b7fea6bc-8f5b-4793-bb20-33e7cd082544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:26.281281 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fea6bc-8f5b-4793-bb20-33e7cd082544 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:26.285008 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155844a-0883-4a35-9e9b-646d21de0408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:26.285594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155844a-0883-4a35-9e9b-646d21de0408 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:26.285868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155844a-0883-4a35-9e9b-646d21de0408 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:26.286134 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155844a-0883-4a35-9e9b-646d21de0408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:26.295507 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155844a-0883-4a35-9e9b-646d21de0408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:26.296957 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155844a-0883-4a35-9e9b-646d21de0408 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:26.306249 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1407/2849] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:54:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:54:26.424451 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd843c0-8520-4b93-a42e-0721168920e6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:26.425833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ddd843c0-8520-4b93-a42e-0721168920e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:26.438622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd843c0-8520-4b93-a42e-0721168920e6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:26.442448 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777252f2-d4fb-42d4-8c1f-1e7c3f4b043b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:26.443039 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777252f2-d4fb-42d4-8c1f-1e7c3f4b043b service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:26.443311 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777252f2-d4fb-42d4-8c1f-1e7c3f4b043b service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:26.443582 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777252f2-d4fb-42d4-8c1f-1e7c3f4b043b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:26.501867 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777252f2-d4fb-42d4-8c1f-1e7c3f4b043b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:26.503331 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777252f2-d4fb-42d4-8c1f-1e7c3f4b043b service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:26.512846 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1443/2850] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:54:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:54:30.419814 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155844a-0883-4a35-9e9b-646d21de0408 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:30.481565 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155844a-0883-4a35-9e9b-646d21de0408 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:30.486098 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:30.486710 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:30.486985 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:30.487254 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] PATH_INFO: `/v3/users/0baeb14fa44940d5890b241ec230e1ff` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:30.488388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0baeb14fa44940d5890b241ec230e1ff)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:30.490283 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:30.640903 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:30.646863 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0baeb14fa44940d5890b241ec230e1ff'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:30.735580 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0baeb14fa44940d5890b241ec230e1ff'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:30.748095 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0baeb14fa44940d5890b241ec230e1ff'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:30.748771 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0baeb14fa44940d5890b241ec230e1ff'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:30.749375 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0baeb14fa44940d5890b241ec230e1ff'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:30.763017 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1408/2851] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:54:30 2022] DELETE /identity/v3/users/0baeb14fa44940d5890b241ec230e1ff => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:30.767525 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9975c-809f-4cf5-8a5b-a49e10855b56 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:30.768306 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9975c-809f-4cf5-8a5b-a49e10855b56 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:30.768416 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9975c-809f-4cf5-8a5b-a49e10855b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:30.789377 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f3b9975c-809f-4cf5-8a5b-a49e10855b56 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:30.790575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f3b9975c-809f-4cf5-8a5b-a49e10855b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:30.802327 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:30.848445 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1444/2852] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:30.857857 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:30.859378 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:30.917320 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b7b60-53bc-4555-81e0-05238c7880c7 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:30.921040 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb9ef2-4b82-45f2-ae45-3cf04eccfc33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:30.921639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb9ef2-4b82-45f2-ae45-3cf04eccfc33 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:30.921920 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb9ef2-4b82-45f2-ae45-3cf04eccfc33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:30.922192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb9ef2-4b82-45f2-ae45-3cf04eccfc33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:30.931076 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb9ef2-4b82-45f2-ae45-3cf04eccfc33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:30.933346 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb9ef2-4b82-45f2-ae45-3cf04eccfc33 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:30.942964 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1409/2853] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:54:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:54:31.388406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9975c-809f-4cf5-8a5b-a49e10855b56 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:31.389932 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f3b9975c-809f-4cf5-8a5b-a49e10855b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:31.402423 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9975c-809f-4cf5-8a5b-a49e10855b56 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:31.407114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:31.407760 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:31.408037 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:31.408363 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] PATH_INFO: `/v3/projects/70a5bebd3c84405897f785c234afc027` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:31.409452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a5bebd3c84405897f785c234afc027)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:31.411369 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:31.522679 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a5bebd3c84405897f785c234afc027', 'request_id': 'req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:31.529354 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a5bebd3c84405897f785c234afc027', 'request_id': 'req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:31.613729 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a5bebd3c84405897f785c234afc027', 'request_id': 'req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:31.624752 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] The token cache is being invalidate because project 70a5bebd3c84405897f785c234afc027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:54:31.625046 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:31.625670 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:31.627183 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1445/2854] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:54:31 2022] DELETE /identity/v3/projects/70a5bebd3c84405897f785c234afc027 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:31.634709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb9ef2-4b82-45f2-ae45-3cf04eccfc33 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:31.641066 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb9ef2-4b82-45f2-ae45-3cf04eccfc33 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:31.697561 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:31.698374 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:31.698666 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:31.698946 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] PATH_INFO: `/v3/users/90f7677a63e5422d926d6deccc46f39c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:31.700131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] RBAC: Authorizing `identity:delete_user(user_id=90f7677a63e5422d926d6deccc46f39c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:31.702050 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:31.914410 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f7677a63e5422d926d6deccc46f39c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:31.975193 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f7677a63e5422d926d6deccc46f39c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:31.981594 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:31.988671 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f7677a63e5422d926d6deccc46f39c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:31.991239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f7677a63e5422d926d6deccc46f39c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:31.991239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f7677a63e5422d926d6deccc46f39c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:32.001873 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1410/2855] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:54:31 2022] DELETE /identity/v3/users/90f7677a63e5422d926d6deccc46f39c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:32.416471 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:32.418668 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:32.434274 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e724d41-dea8-40a1-92b4-f4c2da7d80ef admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:32.493223 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:32.493851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:32.494131 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:32.494406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] PATH_INFO: `/v3/projects/a1122c0735bd4109938e3d1cb0cb7db2` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:32.495530 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1122c0735bd4109938e3d1cb0cb7db2)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:32.497600 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:32.600889 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:32.604470 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1122c0735bd4109938e3d1cb0cb7db2', 'request_id': 'req-a19993a9-0bc4-4b56-96b7-3e757367a917'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:32.611102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1122c0735bd4109938e3d1cb0cb7db2', 'request_id': 'req-a19993a9-0bc4-4b56-96b7-3e757367a917'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:32.689467 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1122c0735bd4109938e3d1cb0cb7db2', 'request_id': 'req-a19993a9-0bc4-4b56-96b7-3e757367a917'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:32.700714 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] The token cache is being invalidate because project a1122c0735bd4109938e3d1cb0cb7db2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:54:32.701216 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:32.702288 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:32.703874 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1446/2856] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:54:32 2022] DELETE /identity/v3/projects/a1122c0735bd4109938e3d1cb0cb7db2 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:32.707855 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:32.709381 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:32.722826 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26116790-5774-45a5-a615-3c6ebede698c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:32.779207 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530c86b-ae9a-4588-ac45-766c8635dde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:32.779811 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530c86b-ae9a-4588-ac45-766c8635dde9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:32.780121 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530c86b-ae9a-4588-ac45-766c8635dde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:32.780397 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530c86b-ae9a-4588-ac45-766c8635dde9 admin admin] PATH_INFO: `/v3/users/668831ec48d742f0a87720f985cbc722` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:32.781513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530c86b-ae9a-4588-ac45-766c8635dde9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=668831ec48d742f0a87720f985cbc722)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:32.783722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530c86b-ae9a-4588-ac45-766c8635dde9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:33.005888 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a530c86b-ae9a-4588-ac45-766c8635dde9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '668831ec48d742f0a87720f985cbc722'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:33.093436 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a530c86b-ae9a-4588-ac45-766c8635dde9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '668831ec48d742f0a87720f985cbc722'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:33.105799 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a530c86b-ae9a-4588-ac45-766c8635dde9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '668831ec48d742f0a87720f985cbc722'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:33.106427 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a530c86b-ae9a-4588-ac45-766c8635dde9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '668831ec48d742f0a87720f985cbc722'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:33.107024 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a530c86b-ae9a-4588-ac45-766c8635dde9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '668831ec48d742f0a87720f985cbc722'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:33.118835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1411/2857] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:54:32 2022] DELETE /identity/v3/users/668831ec48d742f0a87720f985cbc722 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:33.530313 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:33.531905 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:33.545571 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19993a9-0bc4-4b56-96b7-3e757367a917 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:33.603692 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:33.604361 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:33.604644 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:33.605534 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] PATH_INFO: `/v3/projects/be3511d14be343948a23281aaa072f96` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:33.606680 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3511d14be343948a23281aaa072f96)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:33.608626 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:33.722191 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3511d14be343948a23281aaa072f96', 'request_id': 'req-6747d3eb-2080-48e3-87d1-0ddeef983d0a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:33.728299 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3511d14be343948a23281aaa072f96', 'request_id': 'req-6747d3eb-2080-48e3-87d1-0ddeef983d0a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:33.803701 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3511d14be343948a23281aaa072f96', 'request_id': 'req-6747d3eb-2080-48e3-87d1-0ddeef983d0a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:33.814863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] The token cache is being invalidate because project be3511d14be343948a23281aaa072f96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:54:33.818184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:33.818184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6747d3eb-2080-48e3-87d1-0ddeef983d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:33.818184 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1447/2858] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:54:33 2022] DELETE /identity/v3/projects/be3511d14be343948a23281aaa072f96 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:33.823321 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054caa3-67f8-43b5-98d3-0a805aa743bf None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:33.823834 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054caa3-67f8-43b5-98d3-0a805aa743bf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:33.824041 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054caa3-67f8-43b5-98d3-0a805aa743bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:33.843891 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8054caa3-67f8-43b5-98d3-0a805aa743bf None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:33.846904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8054caa3-67f8-43b5-98d3-0a805aa743bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:33.894572 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:33.904045 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1412/2859] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:33.912279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192cd60-8177-447a-a56a-86eec2fe7143 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:33.912569 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192cd60-8177-447a-a56a-86eec2fe7143 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:33.912845 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192cd60-8177-447a-a56a-86eec2fe7143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:33.933083 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f192cd60-8177-447a-a56a-86eec2fe7143 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:33.934309 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f192cd60-8177-447a-a56a-86eec2fe7143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:33.942488 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1448/2860] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:33.946701 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054caa3-67f8-43b5-98d3-0a805aa743bf None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:33.956626 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054caa3-67f8-43b5-98d3-0a805aa743bf None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:33.961367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c7f35-97ec-4d42-b4d1-6e1f67b08432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:33.961962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c7f35-97ec-4d42-b4d1-6e1f67b08432 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:33.962242 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c7f35-97ec-4d42-b4d1-6e1f67b08432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:33.962516 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c7f35-97ec-4d42-b4d1-6e1f67b08432 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:33.963161 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c7f35-97ec-4d42-b4d1-6e1f67b08432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:54:33.963465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c7f35-97ec-4d42-b4d1-6e1f67b08432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:33.965567 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c7f35-97ec-4d42-b4d1-6e1f67b08432 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:33.977552 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1413/2861] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:54:33.982828 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192cd60-8177-447a-a56a-86eec2fe7143 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:33.992272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192cd60-8177-447a-a56a-86eec2fe7143 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:33.997237 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11afc8f-e1cb-47ed-a389-dccfcf53141f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:33.997833 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11afc8f-e1cb-47ed-a389-dccfcf53141f admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:33.998135 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11afc8f-e1cb-47ed-a389-dccfcf53141f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:33.998406 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11afc8f-e1cb-47ed-a389-dccfcf53141f admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:33.999036 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11afc8f-e1cb-47ed-a389-dccfcf53141f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:34.001020 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11afc8f-e1cb-47ed-a389-dccfcf53141f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:34.135398 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1449/2862] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:54:33 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:34.139709 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c7f35-97ec-4d42-b4d1-6e1f67b08432 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:34.150191 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c7f35-97ec-4d42-b4d1-6e1f67b08432 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:34.206583 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7706ca6-745c-4248-8008-483efe3e7237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:34.207183 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7706ca6-745c-4248-8008-483efe3e7237 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:34.207465 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7706ca6-745c-4248-8008-483efe3e7237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:34.207747 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7706ca6-745c-4248-8008-483efe3e7237 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:34.208442 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7706ca6-745c-4248-8008-483efe3e7237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:34.210425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7706ca6-745c-4248-8008-483efe3e7237 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:34.289322 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1414/2863] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:54:34 2022] POST /identity/v3/users => generated 284 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:34.293502 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11afc8f-e1cb-47ed-a389-dccfcf53141f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:34.303868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11afc8f-e1cb-47ed-a389-dccfcf53141f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:34.308513 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55170a30-4c4a-4278-a8df-4901296b2dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:34.309102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55170a30-4c4a-4278-a8df-4901296b2dac admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:34.309378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55170a30-4c4a-4278-a8df-4901296b2dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:34.309651 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55170a30-4c4a-4278-a8df-4901296b2dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:34.310293 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55170a30-4c4a-4278-a8df-4901296b2dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:34.311882 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55170a30-4c4a-4278-a8df-4901296b2dac admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:34.321668 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1450/2864] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:34.326115 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7706ca6-745c-4248-8008-483efe3e7237 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:34.335943 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7706ca6-745c-4248-8008-483efe3e7237 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:34.341197 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8db61-3b55-434a-b4d7-6fd2355a8697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:34.341782 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8db61-3b55-434a-b4d7-6fd2355a8697 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:34.342061 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8db61-3b55-434a-b4d7-6fd2355a8697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:34.342336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8db61-3b55-434a-b4d7-6fd2355a8697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:34.342978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8db61-3b55-434a-b4d7-6fd2355a8697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:34.344579 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8db61-3b55-434a-b4d7-6fd2355a8697 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:34.354942 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1415/2865] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:34.358938 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55170a30-4c4a-4278-a8df-4901296b2dac admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:34.368857 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55170a30-4c4a-4278-a8df-4901296b2dac admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:34.373577 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03136e6-c1b0-4f42-8135-071d6c1744df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:34.374208 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03136e6-c1b0-4f42-8135-071d6c1744df admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:34.374485 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03136e6-c1b0-4f42-8135-071d6c1744df admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:34.374757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03136e6-c1b0-4f42-8135-071d6c1744df admin admin] PATH_INFO: `/v3/projects/703bd8d5efb64c609482f49126933eae/users/9f97a6ae53b6468882a3a62f0e0b839b/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:34.389632 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03136e6-c1b0-4f42-8135-071d6c1744df admin admin] RBAC: Authorizing `identity:create_grant(project_id=703bd8d5efb64c609482f49126933eae, user_id=9f97a6ae53b6468882a3a62f0e0b839b, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:34.392247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03136e6-c1b0-4f42-8135-071d6c1744df admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:34.484997 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:34.490656 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1451/2866] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:34 2022] PUT /identity/v3/projects/703bd8d5efb64c609482f49126933eae/users/9f97a6ae53b6468882a3a62f0e0b839b/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:34.503201 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8921b5-46b0-424b-a2cd-e1e9ad30b0b6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:34.503492 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8921b5-46b0-424b-a2cd-e1e9ad30b0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:34.503766 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8921b5-46b0-424b-a2cd-e1e9ad30b0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:34.532318 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9c8921b5-46b0-424b-a2cd-e1e9ad30b0b6 None None] MFA Rules not processed for user `9f97a6ae53b6468882a3a62f0e0b839b`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:34.533533 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9c8921b5-46b0-424b-a2cd-e1e9ad30b0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:34.596680 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1416/2867] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:34.601272 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03136e6-c1b0-4f42-8135-071d6c1744df admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:34.663137 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03136e6-c1b0-4f42-8135-071d6c1744df admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:34.668167 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c64f8-34d9-44c3-b7be-6dbe0515e86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:34.668763 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c64f8-34d9-44c3-b7be-6dbe0515e86d admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:34.669040 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c64f8-34d9-44c3-b7be-6dbe0515e86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:34.669310 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c64f8-34d9-44c3-b7be-6dbe0515e86d admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:34.669936 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c64f8-34d9-44c3-b7be-6dbe0515e86d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:34.671868 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c64f8-34d9-44c3-b7be-6dbe0515e86d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:34.782820 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1452/2868] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:54:34 2022] POST /identity/v3/projects => generated 367 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:34.786917 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8921b5-46b0-424b-a2cd-e1e9ad30b0b6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:34.798538 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8921b5-46b0-424b-a2cd-e1e9ad30b0b6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:34.844022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:34.852367 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da33e-eef9-463f-beb6-13ad0240d7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:34.852967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da33e-eef9-463f-beb6-13ad0240d7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:34.853239 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da33e-eef9-463f-beb6-13ad0240d7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:34.853513 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da33e-eef9-463f-beb6-13ad0240d7c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:34.854134 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7da33e-eef9-463f-beb6-13ad0240d7c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:34.856868 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7da33e-eef9-463f-beb6-13ad0240d7c3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:34.924097 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1417/2869] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:54:34 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:34.928102 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c64f8-34d9-44c3-b7be-6dbe0515e86d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:34.938728 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c64f8-34d9-44c3-b7be-6dbe0515e86d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:34.943612 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623be79a-637b-4522-b2c5-4b19199488fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:34.944270 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623be79a-637b-4522-b2c5-4b19199488fa admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:34.944547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623be79a-637b-4522-b2c5-4b19199488fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:34.947412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623be79a-637b-4522-b2c5-4b19199488fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:34.947412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623be79a-637b-4522-b2c5-4b19199488fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:34.947412 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623be79a-637b-4522-b2c5-4b19199488fa admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:34.958568 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1453/2870] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 22:54:34.962737 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da33e-eef9-463f-beb6-13ad0240d7c3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:34.973438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da33e-eef9-463f-beb6-13ad0240d7c3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:34.978152 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4d4d9-de73-4bc1-a991-344965f7f3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:34.978805 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d4d9-de73-4bc1-a991-344965f7f3a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:34.979081 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d4d9-de73-4bc1-a991-344965f7f3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:34.979354 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d4d9-de73-4bc1-a991-344965f7f3a9 admin admin] PATH_INFO: `/v3/projects/b04cb9aee8c6483482e0ec36b31d2233/users/05cd260a05a54a02bfd316ca3ff90497/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:34.994721 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4d4d9-de73-4bc1-a991-344965f7f3a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04cb9aee8c6483482e0ec36b31d2233, user_id=05cd260a05a54a02bfd316ca3ff90497, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:34.997494 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4d4d9-de73-4bc1-a991-344965f7f3a9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:35.043926 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1418/2871] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:34 2022] PUT /identity/v3/projects/b04cb9aee8c6483482e0ec36b31d2233/users/05cd260a05a54a02bfd316ca3ff90497/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:35.047897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623be79a-637b-4522-b2c5-4b19199488fa admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:35.058297 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623be79a-637b-4522-b2c5-4b19199488fa admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:35.114322 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e35e9-a7ab-45c8-80b2-06da9148f3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:35.114924 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e35e9-a7ab-45c8-80b2-06da9148f3ef admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:35.115203 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e35e9-a7ab-45c8-80b2-06da9148f3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:35.115477 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e35e9-a7ab-45c8-80b2-06da9148f3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:35.116160 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e35e9-a7ab-45c8-80b2-06da9148f3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:35.117761 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e35e9-a7ab-45c8-80b2-06da9148f3ef admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:35.127408 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1454/2872] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:35.131582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4d4d9-de73-4bc1-a991-344965f7f3a9 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:35.142216 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4d4d9-de73-4bc1-a991-344965f7f3a9 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:35.147085 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e3a8b-ff2e-4a4d-9373-5c7c89179f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:35.147720 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e3a8b-ff2e-4a4d-9373-5c7c89179f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:35.148000 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e3a8b-ff2e-4a4d-9373-5c7c89179f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:35.148303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e3a8b-ff2e-4a4d-9373-5c7c89179f5e admin admin] PATH_INFO: `/v3/projects/b04cb9aee8c6483482e0ec36b31d2233/users/05cd260a05a54a02bfd316ca3ff90497/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:35.150218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e3a8b-ff2e-4a4d-9373-5c7c89179f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04cb9aee8c6483482e0ec36b31d2233, user_id=05cd260a05a54a02bfd316ca3ff90497, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:35.153424 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e3a8b-ff2e-4a4d-9373-5c7c89179f5e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:35.163336 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1419/2873] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:35 2022] PUT /identity/v3/projects/b04cb9aee8c6483482e0ec36b31d2233/users/05cd260a05a54a02bfd316ca3ff90497/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:35.173622 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1b692-cab2-4e22-883d-cebcd3aed7e6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:35.173990 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1b692-cab2-4e22-883d-cebcd3aed7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:35.174320 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1b692-cab2-4e22-883d-cebcd3aed7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:35.202175 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cec1b692-cab2-4e22-883d-cebcd3aed7e6 None None] MFA Rules not processed for user `05cd260a05a54a02bfd316ca3ff90497`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:35.203393 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cec1b692-cab2-4e22-883d-cebcd3aed7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:35.260503 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:35.271086 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1455/2874] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:35.283414 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e3a8b-ff2e-4a4d-9373-5c7c89179f5e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:35.285122 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ba2e3a8b-ff2e-4a4d-9373-5c7c89179f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:35.343345 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e3a8b-ff2e-4a4d-9373-5c7c89179f5e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:35.349510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f92e35-c6b5-4b83-b994-164c21f126ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:35.349510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f92e35-c6b5-4b83-b994-164c21f126ec service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:35.349510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f92e35-c6b5-4b83-b994-164c21f126ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:35.349510 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f92e35-c6b5-4b83-b994-164c21f126ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:35.357652 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f92e35-c6b5-4b83-b994-164c21f126ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:35.359063 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f92e35-c6b5-4b83-b994-164c21f126ec service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:35.368562 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1420/2875] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:54:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:54:35.612698 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1b692-cab2-4e22-883d-cebcd3aed7e6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:35.622610 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1b692-cab2-4e22-883d-cebcd3aed7e6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:35.626180 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8263f5a-0213-44c4-9f6c-2639b65ea675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:35.626770 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8263f5a-0213-44c4-9f6c-2639b65ea675 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:35.627064 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8263f5a-0213-44c4-9f6c-2639b65ea675 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:35.627340 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8263f5a-0213-44c4-9f6c-2639b65ea675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:35.684934 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8263f5a-0213-44c4-9f6c-2639b65ea675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:35.686296 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8263f5a-0213-44c4-9f6c-2639b65ea675 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:35.695072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1456/2876] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:54:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:54:41.852119 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f92e35-c6b5-4b83-b994-164c21f126ec service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:41.913697 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f92e35-c6b5-4b83-b994-164c21f126ec service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:41.918202 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:41.919722 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:41.920022 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:41.920308 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] PATH_INFO: `/v3/users/9f97a6ae53b6468882a3a62f0e0b839b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:41.921417 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f97a6ae53b6468882a3a62f0e0b839b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:41.923445 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:42.114477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f97a6ae53b6468882a3a62f0e0b839b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:42.116977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:42.226221 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f97a6ae53b6468882a3a62f0e0b839b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:42.239977 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f97a6ae53b6468882a3a62f0e0b839b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:42.240663 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f97a6ae53b6468882a3a62f0e0b839b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:42.241257 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f97a6ae53b6468882a3a62f0e0b839b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:42.253962 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1421/2877] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:54:41 2022] DELETE /identity/v3/users/9f97a6ae53b6468882a3a62f0e0b839b => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:42.258509 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11ab37-dbd2-4649-ac93-531a8982be9d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:42.259008 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11ab37-dbd2-4649-ac93-531a8982be9d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:42.259122 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11ab37-dbd2-4649-ac93-531a8982be9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:42.278648 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8b11ab37-dbd2-4649-ac93-531a8982be9d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:42.280004 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8b11ab37-dbd2-4649-ac93-531a8982be9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:42.338773 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1457/2878] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:42.348961 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:42.350394 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:42.409452 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea35468-1d07-4987-96f2-cf64ecadbce6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:42.413004 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64269d-04d6-4d63-9686-9e36a7669c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:42.413601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64269d-04d6-4d63-9686-9e36a7669c7e service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:42.413879 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64269d-04d6-4d63-9686-9e36a7669c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:42.414150 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64269d-04d6-4d63-9686-9e36a7669c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:42.423317 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64269d-04d6-4d63-9686-9e36a7669c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:42.424768 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64269d-04d6-4d63-9686-9e36a7669c7e service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:42.434241 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1422/2879] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:54:42.839897 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11ab37-dbd2-4649-ac93-531a8982be9d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:42.842072 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8b11ab37-dbd2-4649-ac93-531a8982be9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:42.857382 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11ab37-dbd2-4649-ac93-531a8982be9d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:42.859598 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:42.860266 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:42.860547 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:42.860823 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] PATH_INFO: `/v3/projects/703bd8d5efb64c609482f49126933eae` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:42.861975 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=703bd8d5efb64c609482f49126933eae)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:42.864051 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:42.962401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '703bd8d5efb64c609482f49126933eae', 'request_id': 'req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:42.969199 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '703bd8d5efb64c609482f49126933eae', 'request_id': 'req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:43.036246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '703bd8d5efb64c609482f49126933eae', 'request_id': 'req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:43.047306 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] The token cache is being invalidate because project 703bd8d5efb64c609482f49126933eae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:54:43.047601 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:43.048258 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:43.049741 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1458/2880] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:54:42 2022] DELETE /identity/v3/projects/703bd8d5efb64c609482f49126933eae => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:43.053802 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64269d-04d6-4d63-9686-9e36a7669c7e service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:43.064677 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64269d-04d6-4d63-9686-9e36a7669c7e service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:43.119504 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e57cd7-8753-4142-a635-34a31db8aba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:43.120438 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e57cd7-8753-4142-a635-34a31db8aba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:43.120754 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e57cd7-8753-4142-a635-34a31db8aba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:43.121032 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e57cd7-8753-4142-a635-34a31db8aba5 admin admin] PATH_INFO: `/v3/users/05cd260a05a54a02bfd316ca3ff90497` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:43.122664 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e57cd7-8753-4142-a635-34a31db8aba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05cd260a05a54a02bfd316ca3ff90497)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:43.124607 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e57cd7-8753-4142-a635-34a31db8aba5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:43.318172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c6e57cd7-8753-4142-a635-34a31db8aba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05cd260a05a54a02bfd316ca3ff90497'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:43.396285 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c6e57cd7-8753-4142-a635-34a31db8aba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05cd260a05a54a02bfd316ca3ff90497'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:43.408649 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c6e57cd7-8753-4142-a635-34a31db8aba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05cd260a05a54a02bfd316ca3ff90497'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:43.409288 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c6e57cd7-8753-4142-a635-34a31db8aba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05cd260a05a54a02bfd316ca3ff90497'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:43.409896 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c6e57cd7-8753-4142-a635-34a31db8aba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05cd260a05a54a02bfd316ca3ff90497'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:43.411087 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:43.422835 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1423/2881] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:54:43 2022] DELETE /identity/v3/users/05cd260a05a54a02bfd316ca3ff90497 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:43.901323 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:43.902785 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:43.918618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc8a8d-0e0b-4a0f-82fc-a2086a352b86 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:43.932567 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:43.978661 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:43.979417 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:43.979694 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:43.979980 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] PATH_INFO: `/v3/projects/b04cb9aee8c6483482e0ec36b31d2233` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:43.981215 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04cb9aee8c6483482e0ec36b31d2233)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:43.983655 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:44.084187 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04cb9aee8c6483482e0ec36b31d2233', 'request_id': 'req-297a8509-d9b8-45c2-91e9-162ffa2574ed'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:44.091389 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04cb9aee8c6483482e0ec36b31d2233', 'request_id': 'req-297a8509-d9b8-45c2-91e9-162ffa2574ed'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:44.161862 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04cb9aee8c6483482e0ec36b31d2233', 'request_id': 'req-297a8509-d9b8-45c2-91e9-162ffa2574ed'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:44.172575 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] The token cache is being invalidate because project b04cb9aee8c6483482e0ec36b31d2233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:54:44.172883 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:44.173501 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-297a8509-d9b8-45c2-91e9-162ffa2574ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:44.175032 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1459/2882] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:54:43 2022] DELETE /identity/v3/projects/b04cb9aee8c6483482e0ec36b31d2233 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:44.179863 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff4958-4f6a-4ec2-84f3-a8550557c21d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:44.180357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff4958-4f6a-4ec2-84f3-a8550557c21d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:44.180457 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff4958-4f6a-4ec2-84f3-a8550557c21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:44.200870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-61ff4958-4f6a-4ec2-84f3-a8550557c21d None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:44.202094 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-61ff4958-4f6a-4ec2-84f3-a8550557c21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:44.259433 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1424/2883] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:44.268392 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caabec6-a16a-4716-aeb2-d30d5660f311 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:44.268675 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caabec6-a16a-4716-aeb2-d30d5660f311 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:44.268971 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caabec6-a16a-4716-aeb2-d30d5660f311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:44.288822 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7caabec6-a16a-4716-aeb2-d30d5660f311 None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:44.290063 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7caabec6-a16a-4716-aeb2-d30d5660f311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:44.298099 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1460/2884] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:44.302357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff4958-4f6a-4ec2-84f3-a8550557c21d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:44.313003 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff4958-4f6a-4ec2-84f3-a8550557c21d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:44.317529 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae326b-6f8c-4c18-a888-2512ad4148ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:44.318148 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae326b-6f8c-4c18-a888-2512ad4148ec admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:44.318425 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae326b-6f8c-4c18-a888-2512ad4148ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:44.318694 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae326b-6f8c-4c18-a888-2512ad4148ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:44.319331 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae326b-6f8c-4c18-a888-2512ad4148ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 22:54:44.319634 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae326b-6f8c-4c18-a888-2512ad4148ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:44.321282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae326b-6f8c-4c18-a888-2512ad4148ec admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:44.333631 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1425/2885] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 22:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 22:54:44.337754 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caabec6-a16a-4716-aeb2-d30d5660f311 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:44.347112 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caabec6-a16a-4716-aeb2-d30d5660f311 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:44.351650 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ffc37-a3d9-4f8b-a4c5-0c48b07b294d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:44.352264 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ffc37-a3d9-4f8b-a4c5-0c48b07b294d admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:44.352562 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ffc37-a3d9-4f8b-a4c5-0c48b07b294d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:44.352830 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ffc37-a3d9-4f8b-a4c5-0c48b07b294d admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:44.353479 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ffc37-a3d9-4f8b-a4c5-0c48b07b294d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:44.355401 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ffc37-a3d9-4f8b-a4c5-0c48b07b294d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:44.444043 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1461/2886] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:54:44 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:44.448183 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae326b-6f8c-4c18-a888-2512ad4148ec admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:44.459026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae326b-6f8c-4c18-a888-2512ad4148ec admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:44.517400 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf16838-5dc5-4798-8d77-bdc3c301d1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:44.518269 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf16838-5dc5-4798-8d77-bdc3c301d1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:44.518549 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf16838-5dc5-4798-8d77-bdc3c301d1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:44.518822 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf16838-5dc5-4798-8d77-bdc3c301d1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:44.519437 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf16838-5dc5-4798-8d77-bdc3c301d1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:44.521409 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf16838-5dc5-4798-8d77-bdc3c301d1a1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:44.602361 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1426/2887] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:54:44 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:44.606527 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ffc37-a3d9-4f8b-a4c5-0c48b07b294d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:44.616907 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ffc37-a3d9-4f8b-a4c5-0c48b07b294d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:44.621618 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb81a7-31cd-4fdb-adf3-9c1b8bdb5d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:44.622791 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb81a7-31cd-4fdb-adf3-9c1b8bdb5d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:44.623100 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb81a7-31cd-4fdb-adf3-9c1b8bdb5d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:44.623378 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb81a7-31cd-4fdb-adf3-9c1b8bdb5d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:44.624212 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb81a7-31cd-4fdb-adf3-9c1b8bdb5d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:44.625813 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb81a7-31cd-4fdb-adf3-9c1b8bdb5d59 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:44.635712 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1462/2888] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:44.640491 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf16838-5dc5-4798-8d77-bdc3c301d1a1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:44.650956 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf16838-5dc5-4798-8d77-bdc3c301d1a1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:44.656030 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7848a-4a03-4674-b6a2-9aa9383a64b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:44.656642 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7848a-4a03-4674-b6a2-9aa9383a64b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:44.656927 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7848a-4a03-4674-b6a2-9aa9383a64b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:44.657199 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7848a-4a03-4674-b6a2-9aa9383a64b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:44.657842 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7848a-4a03-4674-b6a2-9aa9383a64b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:44.659536 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7848a-4a03-4674-b6a2-9aa9383a64b8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:44.670639 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1427/2889] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:44.674764 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb81a7-31cd-4fdb-adf3-9c1b8bdb5d59 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:44.685649 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb81a7-31cd-4fdb-adf3-9c1b8bdb5d59 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:44.690528 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54457f9d-70b0-4910-8897-966029c61583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:44.691213 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54457f9d-70b0-4910-8897-966029c61583 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:44.691520 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54457f9d-70b0-4910-8897-966029c61583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:44.691826 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54457f9d-70b0-4910-8897-966029c61583 admin admin] PATH_INFO: `/v3/projects/b2badcec03fe4e618276e8c4e24f2922/users/7379ba91cb8a491baa3174f439f84620/roles/87ab4761555e43419d0f23a0f31114f3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:44.706850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54457f9d-70b0-4910-8897-966029c61583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2badcec03fe4e618276e8c4e24f2922, user_id=7379ba91cb8a491baa3174f439f84620, role_id=87ab4761555e43419d0f23a0f31114f3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:44.709560 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54457f9d-70b0-4910-8897-966029c61583 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:44.815851 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1463/2890] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:44 2022] PUT /identity/v3/projects/b2badcec03fe4e618276e8c4e24f2922/users/7379ba91cb8a491baa3174f439f84620/roles/87ab4761555e43419d0f23a0f31114f3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:44.827172 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9dcef-8c6e-457c-bd12-7761805c0752 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:44.827459 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9dcef-8c6e-457c-bd12-7761805c0752 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:44.827761 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9dcef-8c6e-457c-bd12-7761805c0752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:44.855051 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-18e9dcef-8c6e-457c-bd12-7761805c0752 None None] MFA Rules not processed for user `7379ba91cb8a491baa3174f439f84620`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:44.856902 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-18e9dcef-8c6e-457c-bd12-7761805c0752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:44.899675 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:44.920184 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1428/2891] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:44.924583 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54457f9d-70b0-4910-8897-966029c61583 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:44.970239 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:44.988279 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54457f9d-70b0-4910-8897-966029c61583 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:44.993708 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca5350-3962-4d03-ac68-10f6b919ae57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:44.994314 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca5350-3962-4d03-ac68-10f6b919ae57 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:44.994592 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca5350-3962-4d03-ac68-10f6b919ae57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:44.994863 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca5350-3962-4d03-ac68-10f6b919ae57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:44.995483 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca5350-3962-4d03-ac68-10f6b919ae57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:44.999275 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca5350-3962-4d03-ac68-10f6b919ae57 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:45.112585 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1464/2892] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 22:54:44 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:45.116864 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9dcef-8c6e-457c-bd12-7761805c0752 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:45.128303 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9dcef-8c6e-457c-bd12-7761805c0752 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:45.185526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1697690-3686-46b7-8c95-f60a7bcf1ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:45.186218 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1697690-3686-46b7-8c95-f60a7bcf1ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:45.186395 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1697690-3686-46b7-8c95-f60a7bcf1ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:45.186665 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1697690-3686-46b7-8c95-f60a7bcf1ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:45.187284 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1697690-3686-46b7-8c95-f60a7bcf1ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:45.189238 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1697690-3686-46b7-8c95-f60a7bcf1ef9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:45.294477 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1429/2893] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 22:54:45 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 22:54:45.298634 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca5350-3962-4d03-ac68-10f6b919ae57 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:45.309252 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca5350-3962-4d03-ac68-10f6b919ae57 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:45.314029 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4af91-f0c4-4c9e-96ed-6cbefb884f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:45.314631 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4af91-f0c4-4c9e-96ed-6cbefb884f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:45.314912 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4af91-f0c4-4c9e-96ed-6cbefb884f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:45.315189 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4af91-f0c4-4c9e-96ed-6cbefb884f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:45.315843 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4af91-f0c4-4c9e-96ed-6cbefb884f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:45.317608 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4af91-f0c4-4c9e-96ed-6cbefb884f67 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:45.327911 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1465/2894] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:45.331878 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1697690-3686-46b7-8c95-f60a7bcf1ef9 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:45.342467 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1697690-3686-46b7-8c95-f60a7bcf1ef9 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:45.347053 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f64a5-64fe-4433-815d-c252094a06c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:45.347687 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f64a5-64fe-4433-815d-c252094a06c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:45.347967 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f64a5-64fe-4433-815d-c252094a06c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:45.348341 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f64a5-64fe-4433-815d-c252094a06c1 admin admin] PATH_INFO: `/v3/projects/74d4167d498d43a389bb606e65515103/users/526f639e617b44baac4811a1c740677d/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:45.363825 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f64a5-64fe-4433-815d-c252094a06c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d4167d498d43a389bb606e65515103, user_id=526f639e617b44baac4811a1c740677d, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:45.366539 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f64a5-64fe-4433-815d-c252094a06c1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:45.442601 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1430/2895] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:45 2022] PUT /identity/v3/projects/74d4167d498d43a389bb606e65515103/users/526f639e617b44baac4811a1c740677d/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:45.446690 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4af91-f0c4-4c9e-96ed-6cbefb884f67 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:45.457539 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4af91-f0c4-4c9e-96ed-6cbefb884f67 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:45.513856 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758efe5e-8ee1-4276-a6e7-8fc8180f633d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:45.514455 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758efe5e-8ee1-4276-a6e7-8fc8180f633d admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:45.514744 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758efe5e-8ee1-4276-a6e7-8fc8180f633d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:45.515017 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758efe5e-8ee1-4276-a6e7-8fc8180f633d admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:45.515660 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758efe5e-8ee1-4276-a6e7-8fc8180f633d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:45.517287 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758efe5e-8ee1-4276-a6e7-8fc8180f633d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:45.527126 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1466/2896] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 22:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 22:54:45.532182 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f64a5-64fe-4433-815d-c252094a06c1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:45.542518 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f64a5-64fe-4433-815d-c252094a06c1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:45.547559 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843ce81-a492-46c0-b6fc-7843509e37bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:45.548186 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843ce81-a492-46c0-b6fc-7843509e37bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:45.548472 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843ce81-a492-46c0-b6fc-7843509e37bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:45.548751 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843ce81-a492-46c0-b6fc-7843509e37bd admin admin] PATH_INFO: `/v3/projects/74d4167d498d43a389bb606e65515103/users/526f639e617b44baac4811a1c740677d/roles/108664305d2546949f7327b65d4cb667` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:45.550684 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843ce81-a492-46c0-b6fc-7843509e37bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d4167d498d43a389bb606e65515103, user_id=526f639e617b44baac4811a1c740677d, role_id=108664305d2546949f7327b65d4cb667)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:45.553357 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843ce81-a492-46c0-b6fc-7843509e37bd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:45.563620 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1431/2897] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 22:54:45 2022] PUT /identity/v3/projects/74d4167d498d43a389bb606e65515103/users/526f639e617b44baac4811a1c740677d/roles/108664305d2546949f7327b65d4cb667 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:45.573669 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1be30e-738e-47a0-a013-e79fd16fa217 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:45.573965 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1be30e-738e-47a0-a013-e79fd16fa217 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:45.574246 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1be30e-738e-47a0-a013-e79fd16fa217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:45.602453 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9c1be30e-738e-47a0-a013-e79fd16fa217 None None] MFA Rules not processed for user `526f639e617b44baac4811a1c740677d`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:45.603674 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9c1be30e-738e-47a0-a013-e79fd16fa217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:45.671806 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1467/2898] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:45.682478 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843ce81-a492-46c0-b6fc-7843509e37bd admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:45.684183 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1843ce81-a492-46c0-b6fc-7843509e37bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:45.741429 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843ce81-a492-46c0-b6fc-7843509e37bd admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:45.745118 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabcc4a-6bfb-471e-ba33-505d2519a5c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '407b8483d4f34be19d2c7c7b6013b313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:45.745711 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabcc4a-6bfb-471e-ba33-505d2519a5c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:45.745989 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabcc4a-6bfb-471e-ba33-505d2519a5c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:45.746262 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabcc4a-6bfb-471e-ba33-505d2519a5c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:45.793447 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:45.806143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabcc4a-6bfb-471e-ba33-505d2519a5c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:45.806143 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabcc4a-6bfb-471e-ba33-505d2519a5c7 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:45.812870 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1432/2899] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:54:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:54:48.821757 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1be30e-738e-47a0-a013-e79fd16fa217 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:48.862753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:48.883265 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1be30e-738e-47a0-a013-e79fd16fa217 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:48.887774 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:48.888416 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:48.888700 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:48.888987 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] PATH_INFO: `/v3/users/7379ba91cb8a491baa3174f439f84620` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:48.890092 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] RBAC: Authorizing `identity:delete_user(user_id=7379ba91cb8a491baa3174f439f84620)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:48.892025 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:49.169309 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7379ba91cb8a491baa3174f439f84620'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:49.232247 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7379ba91cb8a491baa3174f439f84620'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:49.244985 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7379ba91cb8a491baa3174f439f84620'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:49.252584 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7379ba91cb8a491baa3174f439f84620'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:49.253227 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7379ba91cb8a491baa3174f439f84620'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:49.259302 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1468/2900] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:54:48 2022] DELETE /identity/v3/users/7379ba91cb8a491baa3174f439f84620 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:49.263899 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c3415-ccc0-4439-9bb6-edd10127ecfa None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:49.264479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c3415-ccc0-4439-9bb6-edd10127ecfa None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:49.264479 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c3415-ccc0-4439-9bb6-edd10127ecfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:49.283264 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-541c3415-ccc0-4439-9bb6-edd10127ecfa None None] MFA Rules not processed for user `620b714c359944a6b1d8649f781a6815`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 22:54:49.284504 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-541c3415-ccc0-4439-9bb6-edd10127ecfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:49.341832 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1433/2901] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 22:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 22:54:49.351776 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:49.353211 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:49.412440 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b9160-4baf-4e44-926f-595e8073ffba admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:49.415945 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04342bc-2afd-4a66-b5f4-573b3d578a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baffd4f6a8ad4a6e91d8d9c61cc1dce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:49.416579 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04342bc-2afd-4a66-b5f4-573b3d578a16 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:49.416850 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04342bc-2afd-4a66-b5f4-573b3d578a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:49.417114 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04342bc-2afd-4a66-b5f4-573b3d578a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:49.426156 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04342bc-2afd-4a66-b5f4-573b3d578a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:49.427599 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04342bc-2afd-4a66-b5f4-573b3d578a16 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:49.437125 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1469/2902] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:54:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:54:49.816712 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c3415-ccc0-4439-9bb6-edd10127ecfa None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:49.818139 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-541c3415-ccc0-4439-9bb6-edd10127ecfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:49.830933 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c3415-ccc0-4439-9bb6-edd10127ecfa None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:49.835662 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:49.836354 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:49.836633 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:49.836904 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] PATH_INFO: `/v3/projects/b2badcec03fe4e618276e8c4e24f2922` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:49.838026 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2badcec03fe4e618276e8c4e24f2922)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:49.839978 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:49.923402 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2badcec03fe4e618276e8c4e24f2922', 'request_id': 'req-1f201615-cffa-4461-aa5b-cd385bebb0d8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:49.929410 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2badcec03fe4e618276e8c4e24f2922', 'request_id': 'req-1f201615-cffa-4461-aa5b-cd385bebb0d8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:50.021131 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2badcec03fe4e618276e8c4e24f2922', 'request_id': 'req-1f201615-cffa-4461-aa5b-cd385bebb0d8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:50.033388 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] The token cache is being invalidate because project b2badcec03fe4e618276e8c4e24f2922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:54:50.033785 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:50.034582 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:50.036526 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1434/2903] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:54:49 2022] DELETE /identity/v3/projects/b2badcec03fe4e618276e8c4e24f2922 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:50.040586 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04342bc-2afd-4a66-b5f4-573b3d578a16 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:50.050792 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04342bc-2afd-4a66-b5f4-573b3d578a16 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:50.105617 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:50.106236 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:50.106515 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:50.106788 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] PATH_INFO: `/v3/users/526f639e617b44baac4811a1c740677d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:50.108544 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=526f639e617b44baac4811a1c740677d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:50.110505 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:50.112106 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:50.343452 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '526f639e617b44baac4811a1c740677d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:50.413350 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '526f639e617b44baac4811a1c740677d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:50.426304 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '526f639e617b44baac4811a1c740677d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:50.433035 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '526f639e617b44baac4811a1c740677d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:50.450659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '526f639e617b44baac4811a1c740677d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:50.450659 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1470/2904] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 22:54:50 2022] DELETE /identity/v3/users/526f639e617b44baac4811a1c740677d => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:54:50.845764 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:54:50.847282 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:54:50.861313 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201615-cffa-4461-aa5b-cd385bebb0d8 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:54:50.918524 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620b714c359944a6b1d8649f781a6815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff949bd3888c436789a339dd0a2e16fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:54:50.919152 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:54:50.919430 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:54:50.919705 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] PATH_INFO: `/v3/projects/74d4167d498d43a389bb606e65515103` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:54:50.920981 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d4167d498d43a389bb606e65515103)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:54:50.922881 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:54:50.939113 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:54:51.010399 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d4167d498d43a389bb606e65515103', 'request_id': 'req-86c64c73-bb56-4bf3-92a7-46c905773046'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:51.018384 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d4167d498d43a389bb606e65515103', 'request_id': 'req-86c64c73-bb56-4bf3-92a7-46c905773046'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:51.066255 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d4167d498d43a389bb606e65515103', 'request_id': 'req-86c64c73-bb56-4bf3-92a7-46c905773046'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:51.078244 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] The token cache is being invalidate because project 74d4167d498d43a389bb606e65515103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 22:54:51.078528 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:51.079181 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-86c64c73-bb56-4bf3-92a7-46c905773046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 22:54:51.080686 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1435/2905] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 22:54:50 2022] DELETE /identity/v3/projects/74d4167d498d43a389bb606e65515103 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 22:55:00.436198 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:55:01.074192 dv-cinder-850321-3-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=17,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 22:55:45.632753 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 22:55:45.634165 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:55:45.772742 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26edb37-53a2-4a1e-aa76-275c209eae56 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 22:55:45.831003 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbd501-4b25-45ed-998f-02fd9f78b96d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1d7e39e8c14f5db40efa743fc3b652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a9d6a8b879d4edeaaa815c4556ba6a4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 22:55:45.831608 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbd501-4b25-45ed-998f-02fd9f78b96d service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 22:55:45.831881 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbd501-4b25-45ed-998f-02fd9f78b96d service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 22:55:45.832185 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbd501-4b25-45ed-998f-02fd9f78b96d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 22:55:45.833931 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bffbd501-4b25-45ed-998f-02fd9f78b96d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 22:55:45.845391 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbd501-4b25-45ed-998f-02fd9f78b96d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 22:55:45.846944 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbd501-4b25-45ed-998f-02fd9f78b96d service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 22:55:45.856686 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1471/2906] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 22:55:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 22:55:55.852396 dv-cinder-850321-3-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}